Command line
python /home/saxelrod/Repo/projects/chemprop/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/from_cp_whim_feats_mpnn_from_prc-auc/config.json --data_path /home/saxelrod/rgb_nfs/dsets/cov_2_cl/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'aggregation': 'mean',
 'aggregation_norm': 100,
 'atom_descriptors': None,
 'atom_descriptors_path': None,
 'atom_descriptors_size': 0,
 'atom_features_size': 0,
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/from_cp_whim_feats_mpnn_from_prc-auc/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/rgb_nfs/dsets/cov_2_cl/train_full.csv',
 'dataset_type': 'classification',
 'depth': 5,
 'device': device(type='cuda', index=1),
 'dropout': 0.4,
 'ensemble_size': 1,
 'epochs': 300,
 'extra_metrics': [],
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/from_cp_whim/train_prc-auc.npz'],
 'features_scaling': False,
 'features_size': None,
 'ffn_hidden_size': 300,
 'ffn_num_layers': 3,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 1,
 'grad_clip': None,
 'hidden_size': 300,
 'ignore_columns': None,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'prc-auc',
 'metrics': ['prc-auc'],
 'minimize_score': False,
 'mpn_shared': False,
 'multiclass_num_classes': 3,
 'no_cache_mol': False,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'number_of_molecules': 1,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/from_cp_whim_feats_mpnn_from_prc-auc',
 'save_preds': False,
 'save_smiles_splits': False,
 'seed': 0,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/from_cp_whim/test_prc-auc.npz'],
 'separate_test_path': '/home/saxelrod/rgb_nfs/dsets/cov_2_cl/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/from_cp_whim/val_prc-auc.npz'],
 'separate_val_path': '/home/saxelrod/rgb_nfs/dsets/cov_2_cl/val_full.csv',
 'show_individual_scores': False,
 'smiles_columns': [None],
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': None,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Fold 0
Splitting data with seed 0
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=1728, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=300, out_features=300, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 873,601
Moving model to cuda
Epoch 0
Validation prc-auc = 0.101380
Epoch 1
Validation prc-auc = 0.237798
Epoch 2
Validation prc-auc = 0.331962
Epoch 3
Validation prc-auc = 0.385897
Epoch 4
Loss = 5.9268e-01, PNorm = 40.7433, GNorm = 2.7012, lr_0 = 6.5000e-04
Validation prc-auc = 0.405888
Epoch 5
Validation prc-auc = 0.432981
Epoch 6
Validation prc-auc = 0.417914
Epoch 7
Validation prc-auc = 0.407223
Epoch 8
Validation prc-auc = 0.393581
Epoch 9
Loss = 5.2781e-01, PNorm = 40.8003, GNorm = 2.1336, lr_0 = 9.9743e-04
Validation prc-auc = 0.401991
Epoch 10
Validation prc-auc = 0.414005
Epoch 11
Validation prc-auc = 0.386661
Epoch 12
Validation prc-auc = 0.367464
Epoch 13
Validation prc-auc = 0.358193
Epoch 14
Loss = 6.2182e-01, PNorm = 40.8621, GNorm = 4.7621, lr_0 = 9.8890e-04
Validation prc-auc = 0.358222
Epoch 15
Validation prc-auc = 0.358181
Epoch 16
Validation prc-auc = 0.362890
Epoch 17
Validation prc-auc = 0.352004
Epoch 18
Validation prc-auc = 0.339858
Epoch 19
Loss = 5.2007e-01, PNorm = 40.9182, GNorm = 1.8658, lr_0 = 9.8045e-04
Validation prc-auc = 0.357889
Epoch 20
Validation prc-auc = 0.367628
Epoch 21
Validation prc-auc = 0.355515
Epoch 22
Validation prc-auc = 0.347086
Epoch 23
Validation prc-auc = 0.340746
Epoch 24
Loss = 5.1984e-01, PNorm = 40.9665, GNorm = 2.1482, lr_0 = 9.7207e-04
Validation prc-auc = 0.342563
Epoch 25
Validation prc-auc = 0.341305
Epoch 26
Validation prc-auc = 0.334301
Epoch 27
Validation prc-auc = 0.329340
Epoch 28
Validation prc-auc = 0.300922
Epoch 29
Loss = 4.3866e-01, PNorm = 41.0096, GNorm = 1.9187, lr_0 = 9.6376e-04
Validation prc-auc = 0.306769
Epoch 30
Validation prc-auc = 0.312022
Epoch 31
Validation prc-auc = 0.313389
Epoch 32
Validation prc-auc = 0.312252
Epoch 33
Validation prc-auc = 0.315235
Epoch 34
Loss = 4.3805e-01, PNorm = 41.0570, GNorm = 1.6990, lr_0 = 9.5552e-04
Validation prc-auc = 0.348656
Epoch 35
Validation prc-auc = 0.349288
Epoch 36
Validation prc-auc = 0.340769
Epoch 37
Validation prc-auc = 0.323916
Epoch 38
Validation prc-auc = 0.310508
Epoch 39
Loss = 4.1629e-01, PNorm = 41.1015, GNorm = 1.7117, lr_0 = 9.4735e-04
Validation prc-auc = 0.300143
Epoch 40
Validation prc-auc = 0.301682
Epoch 41
Validation prc-auc = 0.305678
Epoch 42
Validation prc-auc = 0.322029
Epoch 43
Validation prc-auc = 0.327461
Epoch 44
Loss = 3.1706e-01, PNorm = 41.1501, GNorm = 2.0738, lr_0 = 9.3925e-04
Validation prc-auc = 0.366118
Epoch 45
Validation prc-auc = 0.380731
Epoch 46
Validation prc-auc = 0.400146
Epoch 47
Validation prc-auc = 0.411875
Epoch 48
Validation prc-auc = 0.420386
Epoch 49
Loss = 3.5636e-01, PNorm = 41.2027, GNorm = 1.9068, lr_0 = 9.3122e-04
Validation prc-auc = 0.421618
Epoch 50
Validation prc-auc = 0.418746
Epoch 51
Validation prc-auc = 0.401183
Epoch 52
Validation prc-auc = 0.381148
Epoch 53
Validation prc-auc = 0.378307
Epoch 54
Loss = 4.2663e-01, PNorm = 41.2519, GNorm = 1.9116, lr_0 = 9.2326e-04
Validation prc-auc = 0.360441
Epoch 55
Validation prc-auc = 0.340276
Epoch 56
Validation prc-auc = 0.332926
Epoch 57
Validation prc-auc = 0.320971
Epoch 58
Validation prc-auc = 0.299388
Epoch 59
Loss = 3.1436e-01, PNorm = 41.3042, GNorm = 2.0583, lr_0 = 9.1537e-04
Validation prc-auc = 0.305633
Epoch 60
Validation prc-auc = 0.326529
Epoch 61
Validation prc-auc = 0.335058
Epoch 62
Validation prc-auc = 0.345552
Epoch 63
Validation prc-auc = 0.358734
Epoch 64
Loss = 2.7964e-01, PNorm = 41.3683, GNorm = 1.8580, lr_0 = 9.0754e-04
Validation prc-auc = 0.360011
Epoch 65
Validation prc-auc = 0.334163
Epoch 66
Validation prc-auc = 0.317581
Epoch 67
Validation prc-auc = 0.284282
Epoch 68
Validation prc-auc = 0.283913
Epoch 69
Loss = 3.7168e-01, PNorm = 41.4282, GNorm = 2.2497, lr_0 = 8.9978e-04
Validation prc-auc = 0.274358
Epoch 70
Validation prc-auc = 0.277776
Epoch 71
Validation prc-auc = 0.294821
Epoch 72
Validation prc-auc = 0.297226
Epoch 73
Validation prc-auc = 0.312735
Epoch 74
Loss = 3.4861e-01, PNorm = 41.5001, GNorm = 2.1154, lr_0 = 8.9209e-04
Validation prc-auc = 0.313996
Epoch 75
Validation prc-auc = 0.321035
Epoch 76
Validation prc-auc = 0.328466
Epoch 77
Validation prc-auc = 0.336249
Epoch 78
Validation prc-auc = 0.335755
Epoch 79
Loss = 1.9922e-01, PNorm = 41.5749, GNorm = 1.2036, lr_0 = 8.8447e-04
Validation prc-auc = 0.338415
Epoch 80
Validation prc-auc = 0.354274
Epoch 81
Validation prc-auc = 0.369205
Epoch 82
Validation prc-auc = 0.361230
Epoch 83
Validation prc-auc = 0.356365
Epoch 84
Loss = 2.5870e-01, PNorm = 41.6494, GNorm = 1.7214, lr_0 = 8.7691e-04
Validation prc-auc = 0.340227
Epoch 85
Validation prc-auc = 0.321864
Epoch 86
Validation prc-auc = 0.343153
Epoch 87
Validation prc-auc = 0.342742
Epoch 88
Validation prc-auc = 0.364226
Epoch 89
Loss = 2.7267e-01, PNorm = 41.7177, GNorm = 1.3045, lr_0 = 8.6941e-04
Validation prc-auc = 0.350457
Epoch 90
Validation prc-auc = 0.366649
Epoch 91
Validation prc-auc = 0.375449
Epoch 92
Validation prc-auc = 0.379282
Epoch 93
Validation prc-auc = 0.376601
Epoch 94
Loss = 2.6402e-01, PNorm = 41.7856, GNorm = 1.5815, lr_0 = 8.6198e-04
Validation prc-auc = 0.366665
Epoch 95
Validation prc-auc = 0.329079
Epoch 96
Validation prc-auc = 0.336461
Epoch 97
Validation prc-auc = 0.391612
Epoch 98
Validation prc-auc = 0.402467
Epoch 99
Loss = 1.6466e-01, PNorm = 41.8498, GNorm = 1.9307, lr_0 = 8.5461e-04
Validation prc-auc = 0.393366
Epoch 100
Validation prc-auc = 0.397461
Epoch 101
Validation prc-auc = 0.412268
Epoch 102
Validation prc-auc = 0.296231
Epoch 103
Validation prc-auc = 0.272677
Epoch 104
Loss = 1.6893e-01, PNorm = 41.9132, GNorm = 1.4782, lr_0 = 8.4730e-04
Validation prc-auc = 0.260547
Epoch 105
Validation prc-auc = 0.257146
Epoch 106
Validation prc-auc = 0.274456
Epoch 107
Validation prc-auc = 0.337232
Epoch 108
Validation prc-auc = 0.374437
Epoch 109
Loss = 2.0943e-01, PNorm = 41.9806, GNorm = 1.3373, lr_0 = 8.4006e-04
Validation prc-auc = 0.375008
Epoch 110
Validation prc-auc = 0.381392
Epoch 111
Validation prc-auc = 0.391876
Epoch 112
Validation prc-auc = 0.387455
Epoch 113
Validation prc-auc = 0.360547
Epoch 114
Loss = 2.3862e-01, PNorm = 42.0486, GNorm = 1.5078, lr_0 = 8.3288e-04
Validation prc-auc = 0.294732
Epoch 115
Validation prc-auc = 0.254545
Epoch 116
Validation prc-auc = 0.275871
Epoch 117
Validation prc-auc = 0.323570
Epoch 118
Validation prc-auc = 0.345538
Epoch 119
Loss = 1.1808e-01, PNorm = 42.1129, GNorm = 1.2069, lr_0 = 8.2576e-04
Validation prc-auc = 0.345141
Epoch 120
Validation prc-auc = 0.342748
Epoch 121
Validation prc-auc = 0.330529
Epoch 122
Validation prc-auc = 0.327447
Epoch 123
Validation prc-auc = 0.310507
Epoch 124
Loss = 2.4508e-01, PNorm = 42.1741, GNorm = 2.0959, lr_0 = 8.1870e-04
Validation prc-auc = 0.298284
Epoch 125
Validation prc-auc = 0.287241
Epoch 126
Validation prc-auc = 0.273486
Epoch 127
Validation prc-auc = 0.285169
Epoch 128
Validation prc-auc = 0.309547
Epoch 129
Loss = 2.3608e-01, PNorm = 42.2387, GNorm = 2.6063, lr_0 = 8.1170e-04
Validation prc-auc = 0.342762
Epoch 130
Validation prc-auc = 0.360567
Epoch 131
Validation prc-auc = 0.376192
Epoch 132
Validation prc-auc = 0.372920
Epoch 133
Validation prc-auc = 0.366513
Epoch 134
Loss = 1.2662e-01, PNorm = 42.3125, GNorm = 1.4202, lr_0 = 8.0476e-04
Validation prc-auc = 0.366238
Epoch 135
Validation prc-auc = 0.374740
Epoch 136
Validation prc-auc = 0.382648
Epoch 137
Validation prc-auc = 0.380381
Epoch 138
Validation prc-auc = 0.363474
Epoch 139
Loss = 9.7306e-02, PNorm = 42.3739, GNorm = 0.7592, lr_0 = 7.9788e-04
Validation prc-auc = 0.359051
Epoch 140
Validation prc-auc = 0.337133
Epoch 141
Validation prc-auc = 0.342032
Epoch 142
Validation prc-auc = 0.341744
Epoch 143
Validation prc-auc = 0.328277
Epoch 144
Loss = 1.7069e-01, PNorm = 42.4289, GNorm = 1.6734, lr_0 = 7.9106e-04
Validation prc-auc = 0.302825
Epoch 145
Validation prc-auc = 0.298968
Epoch 146
Validation prc-auc = 0.300173
Epoch 147
Validation prc-auc = 0.299644
Epoch 148
Validation prc-auc = 0.348101
Epoch 149
Loss = 1.7055e-01, PNorm = 42.4878, GNorm = 2.1737, lr_0 = 7.8430e-04
Validation prc-auc = 0.337725
Epoch 150
Validation prc-auc = 0.340061
Epoch 151
Validation prc-auc = 0.342297
Epoch 152
Validation prc-auc = 0.334435
Epoch 153
Validation prc-auc = 0.315927
Epoch 154
Loss = 1.4316e-01, PNorm = 42.5415, GNorm = 0.5616, lr_0 = 7.7759e-04
Validation prc-auc = 0.311416
Epoch 155
Validation prc-auc = 0.307649
Epoch 156
Validation prc-auc = 0.307416
Epoch 157
Validation prc-auc = 0.358385
Epoch 158
Validation prc-auc = 0.361998
Epoch 159
Loss = 1.6485e-01, PNorm = 42.6010, GNorm = 1.7828, lr_0 = 7.7095e-04
Validation prc-auc = 0.331159
Epoch 160
Validation prc-auc = 0.335388
Epoch 161
Validation prc-auc = 0.336030
Epoch 162
Validation prc-auc = 0.371443
Epoch 163
Validation prc-auc = 0.337847
Epoch 164
Loss = 1.3723e-01, PNorm = 42.6524, GNorm = 1.8004, lr_0 = 7.6436e-04
Validation prc-auc = 0.335808
Epoch 165
Validation prc-auc = 0.325144
Epoch 166
Validation prc-auc = 0.286134
Epoch 167
Validation prc-auc = 0.312015
Epoch 168
Validation prc-auc = 0.305965
Epoch 169
Loss = 8.1206e-02, PNorm = 42.7077, GNorm = 0.7217, lr_0 = 7.5782e-04
Validation prc-auc = 0.335965
Epoch 170
Validation prc-auc = 0.353292
Epoch 171
Validation prc-auc = 0.355254
Epoch 172
Validation prc-auc = 0.349860
Epoch 173
Validation prc-auc = 0.333170
Epoch 174
Loss = 2.5314e-01, PNorm = 42.7648, GNorm = 1.7387, lr_0 = 7.5134e-04
Validation prc-auc = 0.302153
Epoch 175
Validation prc-auc = 0.282235
Epoch 176
Validation prc-auc = 0.275966
Epoch 177
Validation prc-auc = 0.275132
Epoch 178
Validation prc-auc = 0.310630
Epoch 179
Loss = 1.8463e-01, PNorm = 42.8258, GNorm = 2.0649, lr_0 = 7.4492e-04
Validation prc-auc = 0.326469
Epoch 180
Validation prc-auc = 0.298276
Epoch 181
Validation prc-auc = 0.290285
Epoch 182
Validation prc-auc = 0.300328
Epoch 183
Validation prc-auc = 0.306201
Epoch 184
Loss = 1.4587e-01, PNorm = 42.8900, GNorm = 1.1915, lr_0 = 7.3855e-04
Validation prc-auc = 0.313535
Epoch 185
Validation prc-auc = 0.323484
Epoch 186
Validation prc-auc = 0.323823
Epoch 187
Validation prc-auc = 0.322078
Epoch 188
Validation prc-auc = 0.309913
Epoch 189
Loss = 8.9416e-02, PNorm = 42.9392, GNorm = 1.1677, lr_0 = 7.3224e-04
Validation prc-auc = 0.289085
Epoch 190
Validation prc-auc = 0.288558
Epoch 191
Validation prc-auc = 0.283890
Epoch 192
Validation prc-auc = 0.292658
Epoch 193
Validation prc-auc = 0.302980
Epoch 194
Loss = 1.4615e-01, PNorm = 42.9926, GNorm = 1.0587, lr_0 = 7.2598e-04
Validation prc-auc = 0.299932
Epoch 195
Validation prc-auc = 0.299151
Epoch 196
Validation prc-auc = 0.299780
Epoch 197
Validation prc-auc = 0.305868
Epoch 198
Validation prc-auc = 0.309065
Epoch 199
Loss = 1.2344e-01, PNorm = 43.0457, GNorm = 1.3819, lr_0 = 7.1977e-04
Validation prc-auc = 0.306717
Epoch 200
Validation prc-auc = 0.310528
Epoch 201
Validation prc-auc = 0.311752
Epoch 202
Validation prc-auc = 0.302772
Epoch 203
Validation prc-auc = 0.301946
Epoch 204
Loss = 5.7364e-02, PNorm = 43.1072, GNorm = 1.6010, lr_0 = 7.1362e-04
Validation prc-auc = 0.309228
Epoch 205
Validation prc-auc = 0.315003
Epoch 206
Validation prc-auc = 0.313875
Epoch 207
Validation prc-auc = 0.326103
Epoch 208
Validation prc-auc = 0.332392
Epoch 209
Loss = 1.4495e-01, PNorm = 43.1563, GNorm = 1.3743, lr_0 = 7.0752e-04
Validation prc-auc = 0.348586
Epoch 210
Validation prc-auc = 0.331413
Epoch 211
Validation prc-auc = 0.308554
Epoch 212
Validation prc-auc = 0.294290
Epoch 213
Validation prc-auc = 0.284773
Epoch 214
Loss = 1.4562e-01, PNorm = 43.2047, GNorm = 3.9301, lr_0 = 7.0147e-04
Validation prc-auc = 0.283674
Epoch 215
Validation prc-auc = 0.299487
Epoch 216
Validation prc-auc = 0.320763
Epoch 217
Validation prc-auc = 0.325786
Epoch 218
Validation prc-auc = 0.342123
Epoch 219
Loss = 1.5090e-01, PNorm = 43.2615, GNorm = 1.8122, lr_0 = 6.9548e-04
Validation prc-auc = 0.350722
Epoch 220
Validation prc-auc = 0.372853
Epoch 221
Validation prc-auc = 0.343882
Epoch 222
Validation prc-auc = 0.340716
Epoch 223
Validation prc-auc = 0.338947
Epoch 224
Loss = 1.0620e-01, PNorm = 43.3192, GNorm = 1.4226, lr_0 = 6.8953e-04
Validation prc-auc = 0.324069
Epoch 225
Validation prc-auc = 0.315901
Epoch 226
Validation prc-auc = 0.303185
Epoch 227
Validation prc-auc = 0.298894
Epoch 228
Validation prc-auc = 0.291038
Epoch 229
Loss = 1.7224e-01, PNorm = 43.3653, GNorm = 2.2511, lr_0 = 6.8364e-04
Validation prc-auc = 0.287522
Epoch 230
Validation prc-auc = 0.287639
Epoch 231
Validation prc-auc = 0.287650
Epoch 232
Validation prc-auc = 0.292524
Epoch 233
Validation prc-auc = 0.296266
Epoch 234
Loss = 1.4528e-01, PNorm = 43.4125, GNorm = 2.2917, lr_0 = 6.7779e-04
Validation prc-auc = 0.309856
Epoch 235
Validation prc-auc = 0.320628
Epoch 236
Validation prc-auc = 0.336135
Epoch 237
Validation prc-auc = 0.351500
Epoch 238
Validation prc-auc = 0.370722
Epoch 239
Loss = 8.0287e-02, PNorm = 43.4568, GNorm = 0.8292, lr_0 = 6.7200e-04
Validation prc-auc = 0.370066
Epoch 240
Validation prc-auc = 0.369681
Epoch 241
Validation prc-auc = 0.356453
Epoch 242
Validation prc-auc = 0.334285
Epoch 243
Validation prc-auc = 0.323825
Epoch 244
Loss = 1.2400e-01, PNorm = 43.4959, GNorm = 0.8280, lr_0 = 6.6625e-04
Validation prc-auc = 0.312896
Epoch 245
Validation prc-auc = 0.304207
Epoch 246
Validation prc-auc = 0.300112
Epoch 247
Validation prc-auc = 0.295514
Epoch 248
Validation prc-auc = 0.292822
Epoch 249
Loss = 1.0253e-01, PNorm = 43.5315, GNorm = 1.1106, lr_0 = 6.6056e-04
Validation prc-auc = 0.295691
Epoch 250
Validation prc-auc = 0.293874
Epoch 251
Validation prc-auc = 0.287405
Epoch 252
Validation prc-auc = 0.290314
Epoch 253
Validation prc-auc = 0.305492
Epoch 254
Loss = 1.1125e-01, PNorm = 43.5802, GNorm = 1.4741, lr_0 = 6.5491e-04
Validation prc-auc = 0.308896
Epoch 255
Validation prc-auc = 0.319704
Epoch 256
Validation prc-auc = 0.330712
Epoch 257
Validation prc-auc = 0.323937
Epoch 258
Validation prc-auc = 0.310006
Epoch 259
Loss = 1.0324e-01, PNorm = 43.6233, GNorm = 2.2256, lr_0 = 6.4931e-04
Validation prc-auc = 0.301571
Epoch 260
Validation prc-auc = 0.293465
Epoch 261
Validation prc-auc = 0.289529
Epoch 262
Validation prc-auc = 0.295874
Epoch 263
Validation prc-auc = 0.312607
Epoch 264
Loss = 1.3703e-01, PNorm = 43.6639, GNorm = 1.7716, lr_0 = 6.4376e-04
Validation prc-auc = 0.311977
Epoch 265
Validation prc-auc = 0.307469
Epoch 266
Validation prc-auc = 0.303857
Epoch 267
Validation prc-auc = 0.305244
Epoch 268
Validation prc-auc = 0.299749
Epoch 269
Loss = 3.5751e-02, PNorm = 43.7090, GNorm = 0.4562, lr_0 = 6.3826e-04
Validation prc-auc = 0.291664
Epoch 270
Validation prc-auc = 0.282735
Epoch 271
Validation prc-auc = 0.280683
Epoch 272
Validation prc-auc = 0.286553
Epoch 273
Validation prc-auc = 0.291880
Epoch 274
Loss = 7.7967e-02, PNorm = 43.7534, GNorm = 1.3406, lr_0 = 6.3280e-04
Validation prc-auc = 0.304228
Epoch 275
Validation prc-auc = 0.316687
Epoch 276
Validation prc-auc = 0.323561
Epoch 277
Validation prc-auc = 0.322476
Epoch 278
Validation prc-auc = 0.317659
Epoch 279
Loss = 4.9051e-02, PNorm = 43.7834, GNorm = 0.1989, lr_0 = 6.2739e-04
Validation prc-auc = 0.321215
Epoch 280
Validation prc-auc = 0.329635
Epoch 281
Validation prc-auc = 0.335615
Epoch 282
Validation prc-auc = 0.322857
Epoch 283
Validation prc-auc = 0.312731
Epoch 284
Loss = 8.9523e-02, PNorm = 43.8114, GNorm = 1.7384, lr_0 = 6.2203e-04
Validation prc-auc = 0.308285
Epoch 285
Validation prc-auc = 0.308005
Epoch 286
Validation prc-auc = 0.303721
Epoch 287
Validation prc-auc = 0.302614
Epoch 288
Validation prc-auc = 0.303931
Epoch 289
Loss = 5.4032e-02, PNorm = 43.8461, GNorm = 0.7717, lr_0 = 6.1671e-04
Validation prc-auc = 0.298755
Epoch 290
Validation prc-auc = 0.296926
Epoch 291
Validation prc-auc = 0.299625
Epoch 292
Validation prc-auc = 0.301587
Epoch 293
Validation prc-auc = 0.310890
Epoch 294
Loss = 5.7391e-02, PNorm = 43.8777, GNorm = 0.2423, lr_0 = 6.1144e-04
Validation prc-auc = 0.339396
Epoch 295
Validation prc-auc = 0.361733
Epoch 296
Validation prc-auc = 0.375270
Epoch 297
Validation prc-auc = 0.416296
Epoch 298
Validation prc-auc = 0.412112
Epoch 299
Loss = 8.2882e-02, PNorm = 43.9132, GNorm = 1.2996, lr_0 = 6.0621e-04
Validation prc-auc = 0.389098
Model 0 best validation prc-auc = 0.432981 on epoch 5
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test prc-auc = 0.399693
Ensemble test prc-auc = 0.399693
Fold 1
Splitting data with seed 1
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=1728, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=300, out_features=300, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 873,601
Moving model to cuda
Epoch 0
Validation prc-auc = 0.095856
Epoch 1
Validation prc-auc = 0.175053
Epoch 2
Validation prc-auc = 0.417590
Epoch 3
Validation prc-auc = 0.406528
Epoch 4
Loss = 6.1871e-01, PNorm = 40.7443, GNorm = 4.4669, lr_0 = 6.5000e-04
Validation prc-auc = 0.392333
Epoch 5
Validation prc-auc = 0.413106
Epoch 6
Validation prc-auc = 0.421716
Epoch 7
Validation prc-auc = 0.400206
Epoch 8
Validation prc-auc = 0.393174
Epoch 9
Loss = 6.0271e-01, PNorm = 40.8048, GNorm = 2.6897, lr_0 = 9.9743e-04
Validation prc-auc = 0.385231
Epoch 10
Validation prc-auc = 0.358402
Epoch 11
Validation prc-auc = 0.332652
Epoch 12
Validation prc-auc = 0.336871
Epoch 13
Validation prc-auc = 0.340020
Epoch 14
Loss = 5.4995e-01, PNorm = 40.8696, GNorm = 2.2401, lr_0 = 9.8890e-04
Validation prc-auc = 0.344270
Epoch 15
Validation prc-auc = 0.344053
Epoch 16
Validation prc-auc = 0.345821
Epoch 17
Validation prc-auc = 0.345351
Epoch 18
Validation prc-auc = 0.349014
Epoch 19
Loss = 5.1258e-01, PNorm = 40.9251, GNorm = 1.9623, lr_0 = 9.8045e-04
Validation prc-auc = 0.366573
Epoch 20
Validation prc-auc = 0.374464
Epoch 21
Validation prc-auc = 0.366714
Epoch 22
Validation prc-auc = 0.351204
Epoch 23
Validation prc-auc = 0.349562
Epoch 24
Loss = 4.9757e-01, PNorm = 40.9773, GNorm = 2.0358, lr_0 = 9.7207e-04
Validation prc-auc = 0.346967
Epoch 25
Validation prc-auc = 0.348120
Epoch 26
Validation prc-auc = 0.341341
Epoch 27
Validation prc-auc = 0.335670
Epoch 28
Validation prc-auc = 0.325751
Epoch 29
Loss = 4.8371e-01, PNorm = 41.0184, GNorm = 1.4172, lr_0 = 9.6376e-04
Validation prc-auc = 0.316785
Epoch 30
Validation prc-auc = 0.314112
Epoch 31
Validation prc-auc = 0.313088
Epoch 32
Validation prc-auc = 0.313727
Epoch 33
Validation prc-auc = 0.318528
Epoch 34
Loss = 4.5052e-01, PNorm = 41.0615, GNorm = 1.5149, lr_0 = 9.5552e-04
Validation prc-auc = 0.323211
Epoch 35
Validation prc-auc = 0.324327
Epoch 36
Validation prc-auc = 0.328688
Epoch 37
Validation prc-auc = 0.337119
Epoch 38
Validation prc-auc = 0.342029
Epoch 39
Loss = 4.6441e-01, PNorm = 41.1044, GNorm = 1.7016, lr_0 = 9.4735e-04
Validation prc-auc = 0.339649
Epoch 40
Validation prc-auc = 0.339936
Epoch 41
Validation prc-auc = 0.341067
Epoch 42
Validation prc-auc = 0.374986
Epoch 43
Validation prc-auc = 0.379898
Epoch 44
Loss = 3.9475e-01, PNorm = 41.1477, GNorm = 1.7548, lr_0 = 9.3925e-04
Validation prc-auc = 0.382894
Epoch 45
Validation prc-auc = 0.383097
Epoch 46
Validation prc-auc = 0.386107
Epoch 47
Validation prc-auc = 0.391931
Epoch 48
Validation prc-auc = 0.379286
Epoch 49
Loss = 3.3519e-01, PNorm = 41.1951, GNorm = 1.3383, lr_0 = 9.3122e-04
Validation prc-auc = 0.372916
Epoch 50
Validation prc-auc = 0.376636
Epoch 51
Validation prc-auc = 0.371879
Epoch 52
Validation prc-auc = 0.370362
Epoch 53
Validation prc-auc = 0.381979
Epoch 54
Loss = 4.5263e-01, PNorm = 41.2472, GNorm = 2.4665, lr_0 = 9.2326e-04
Validation prc-auc = 0.375707
Epoch 55
Validation prc-auc = 0.368148
Epoch 56
Validation prc-auc = 0.370733
Epoch 57
Validation prc-auc = 0.367544
Epoch 58
Validation prc-auc = 0.364462
Epoch 59
Loss = 3.9634e-01, PNorm = 41.2967, GNorm = 5.2102, lr_0 = 9.1537e-04
Validation prc-auc = 0.364157
Epoch 60
Validation prc-auc = 0.362369
Epoch 61
Validation prc-auc = 0.324717
Epoch 62
Validation prc-auc = 0.319512
Epoch 63
Validation prc-auc = 0.315637
Epoch 64
Loss = 2.9069e-01, PNorm = 41.3566, GNorm = 1.2450, lr_0 = 9.0754e-04
Validation prc-auc = 0.314736
Epoch 65
Validation prc-auc = 0.316969
Epoch 66
Validation prc-auc = 0.310314
Epoch 67
Validation prc-auc = 0.285054
Epoch 68
Validation prc-auc = 0.288202
Epoch 69
Loss = 3.7069e-01, PNorm = 41.4169, GNorm = 2.0223, lr_0 = 8.9978e-04
Validation prc-auc = 0.290287
Epoch 70
Validation prc-auc = 0.308275
Epoch 71
Validation prc-auc = 0.332757
Epoch 72
Validation prc-auc = 0.347212
Epoch 73
Validation prc-auc = 0.347457
Epoch 74
Loss = 3.3572e-01, PNorm = 41.4756, GNorm = 1.8310, lr_0 = 8.9209e-04
Validation prc-auc = 0.338898
Epoch 75
Validation prc-auc = 0.339132
Epoch 76
Validation prc-auc = 0.331519
Epoch 77
Validation prc-auc = 0.313437
Epoch 78
Validation prc-auc = 0.310563
Epoch 79
Loss = 2.9454e-01, PNorm = 41.5351, GNorm = 1.6952, lr_0 = 8.8447e-04
Validation prc-auc = 0.309777
Epoch 80
Validation prc-auc = 0.302003
Epoch 81
Validation prc-auc = 0.306701
Epoch 82
Validation prc-auc = 0.312988
Epoch 83
Validation prc-auc = 0.317036
Epoch 84
Loss = 2.3314e-01, PNorm = 41.6013, GNorm = 1.4048, lr_0 = 8.7691e-04
Validation prc-auc = 0.329511
Epoch 85
Validation prc-auc = 0.330472
Epoch 86
Validation prc-auc = 0.330957
Epoch 87
Validation prc-auc = 0.364615
Epoch 88
Validation prc-auc = 0.343212
Epoch 89
Loss = 2.6845e-01, PNorm = 41.6704, GNorm = 1.4023, lr_0 = 8.6941e-04
Validation prc-auc = 0.356299
Epoch 90
Validation prc-auc = 0.355710
Epoch 91
Validation prc-auc = 0.342200
Epoch 92
Validation prc-auc = 0.336647
Epoch 93
Validation prc-auc = 0.356920
Epoch 94
Loss = 3.5605e-01, PNorm = 41.7406, GNorm = 8.1793, lr_0 = 8.6198e-04
Validation prc-auc = 0.357734
Epoch 95
Validation prc-auc = 0.323406
Epoch 96
Validation prc-auc = 0.322734
Epoch 97
Validation prc-auc = 0.332431
Epoch 98
Validation prc-auc = 0.360127
Epoch 99
Loss = 2.1497e-01, PNorm = 41.8181, GNorm = 2.3649, lr_0 = 8.5461e-04
Validation prc-auc = 0.337668
Epoch 100
Validation prc-auc = 0.336380
Epoch 101
Validation prc-auc = 0.323637
Epoch 102
Validation prc-auc = 0.330486
Epoch 103
Validation prc-auc = 0.334241
Epoch 104
Loss = 1.8659e-01, PNorm = 41.8860, GNorm = 1.3781, lr_0 = 8.4730e-04
Validation prc-auc = 0.346460
Epoch 105
Validation prc-auc = 0.339195
Epoch 106
Validation prc-auc = 0.337197
Epoch 107
Validation prc-auc = 0.335929
Epoch 108
Validation prc-auc = 0.344955
Epoch 109
Loss = 2.0619e-01, PNorm = 41.9500, GNorm = 1.8882, lr_0 = 8.4006e-04
Validation prc-auc = 0.352233
Epoch 110
Validation prc-auc = 0.355656
Epoch 111
Validation prc-auc = 0.356818
Epoch 112
Validation prc-auc = 0.343956
Epoch 113
Validation prc-auc = 0.317187
Epoch 114
Loss = 1.7998e-01, PNorm = 42.0166, GNorm = 1.5098, lr_0 = 8.3288e-04
Validation prc-auc = 0.303576
Epoch 115
Validation prc-auc = 0.287217
Epoch 116
Validation prc-auc = 0.287548
Epoch 117
Validation prc-auc = 0.294119
Epoch 118
Validation prc-auc = 0.301521
Epoch 119
Loss = 1.7488e-01, PNorm = 42.0665, GNorm = 1.3977, lr_0 = 8.2576e-04
Validation prc-auc = 0.300749
Epoch 120
Validation prc-auc = 0.299219
Epoch 121
Validation prc-auc = 0.298010
Epoch 122
Validation prc-auc = 0.300434
Epoch 123
Validation prc-auc = 0.299849
Epoch 124
Loss = 2.0010e-01, PNorm = 42.1285, GNorm = 1.5984, lr_0 = 8.1870e-04
Validation prc-auc = 0.302060
Epoch 125
Validation prc-auc = 0.313806
Epoch 126
Validation prc-auc = 0.340221
Epoch 127
Validation prc-auc = 0.348217
Epoch 128
Validation prc-auc = 0.338325
Epoch 129
Loss = 1.5554e-01, PNorm = 42.1996, GNorm = 1.3960, lr_0 = 8.1170e-04
Validation prc-auc = 0.339175
Epoch 130
Validation prc-auc = 0.344740
Epoch 131
Validation prc-auc = 0.346833
Epoch 132
Validation prc-auc = 0.343270
Epoch 133
Validation prc-auc = 0.336675
Epoch 134
Loss = 2.0082e-01, PNorm = 42.2647, GNorm = 1.4009, lr_0 = 8.0476e-04
Validation prc-auc = 0.291509
Epoch 135
Validation prc-auc = 0.275792
Epoch 136
Validation prc-auc = 0.278517
Epoch 137
Validation prc-auc = 0.279597
Epoch 138
Validation prc-auc = 0.286692
Epoch 139
Loss = 1.6020e-01, PNorm = 42.3301, GNorm = 1.9435, lr_0 = 7.9788e-04
Validation prc-auc = 0.287820
Epoch 140
Validation prc-auc = 0.298255
Epoch 141
Validation prc-auc = 0.304615
Epoch 142
Validation prc-auc = 0.316954
Epoch 143
Validation prc-auc = 0.324959
Epoch 144
Loss = 1.4646e-01, PNorm = 42.3972, GNorm = 1.2704, lr_0 = 7.9106e-04
Validation prc-auc = 0.325796
Epoch 145
Validation prc-auc = 0.327004
Epoch 146
Validation prc-auc = 0.334863
Epoch 147
Validation prc-auc = 0.338628
Epoch 148
Validation prc-auc = 0.331517
Epoch 149
Loss = 1.5075e-01, PNorm = 42.4522, GNorm = 1.7126, lr_0 = 7.8430e-04
Validation prc-auc = 0.323644
Epoch 150
Validation prc-auc = 0.331782
Epoch 151
Validation prc-auc = 0.315364
Epoch 152
Validation prc-auc = 0.297459
Epoch 153
Validation prc-auc = 0.295717
Epoch 154
Loss = 9.9540e-02, PNorm = 42.5107, GNorm = 1.4843, lr_0 = 7.7759e-04
Validation prc-auc = 0.303518
Epoch 155
Validation prc-auc = 0.314587
Epoch 156
Validation prc-auc = 0.336690
Epoch 157
Validation prc-auc = 0.364619
Epoch 158
Validation prc-auc = 0.406477
Epoch 159
Loss = 2.1734e-01, PNorm = 42.5746, GNorm = 2.0100, lr_0 = 7.7095e-04
Validation prc-auc = 0.409144
Epoch 160
Validation prc-auc = 0.394564
Epoch 161
Validation prc-auc = 0.398374
Epoch 162
Validation prc-auc = 0.425651
Epoch 163
Validation prc-auc = 0.399913
Epoch 164
Loss = 1.1823e-01, PNorm = 42.6356, GNorm = 1.0555, lr_0 = 7.6436e-04
Validation prc-auc = 0.344817
Epoch 165
Validation prc-auc = 0.331681
Epoch 166
Validation prc-auc = 0.325492
Epoch 167
Validation prc-auc = 0.328466
Epoch 168
Validation prc-auc = 0.321783
Epoch 169
Loss = 1.2831e-01, PNorm = 42.6993, GNorm = 1.4110, lr_0 = 7.5782e-04
Validation prc-auc = 0.318498
Epoch 170
Validation prc-auc = 0.309083
Epoch 171
Validation prc-auc = 0.306459
Epoch 172
Validation prc-auc = 0.301995
Epoch 173
Validation prc-auc = 0.312252
Epoch 174
Loss = 1.8143e-01, PNorm = 42.7636, GNorm = 1.4227, lr_0 = 7.5134e-04
Validation prc-auc = 0.316623
Epoch 175
Validation prc-auc = 0.328918
Epoch 176
Validation prc-auc = 0.327065
Epoch 177
Validation prc-auc = 0.317872
Epoch 178
Validation prc-auc = 0.322465
Epoch 179
Loss = 1.1918e-01, PNorm = 42.8175, GNorm = 1.6629, lr_0 = 7.4492e-04
Validation prc-auc = 0.318275
Epoch 180
Validation prc-auc = 0.291697
Epoch 181
Validation prc-auc = 0.291439
Epoch 182
Validation prc-auc = 0.285796
Epoch 183
Validation prc-auc = 0.289117
Epoch 184
Loss = 1.2542e-01, PNorm = 42.8698, GNorm = 1.7698, lr_0 = 7.3855e-04
Validation prc-auc = 0.297659
Epoch 185
Validation prc-auc = 0.326663
Epoch 186
Validation prc-auc = 0.406598
Epoch 187
Validation prc-auc = 0.408715
Epoch 188
Validation prc-auc = 0.390103
Epoch 189
Loss = 2.4280e-01, PNorm = 42.9158, GNorm = 1.4351, lr_0 = 7.3224e-04
Validation prc-auc = 0.381811
Epoch 190
Validation prc-auc = 0.385158
Epoch 191
Validation prc-auc = 0.396406
Epoch 192
Validation prc-auc = 0.391322
Epoch 193
Validation prc-auc = 0.403029
Epoch 194
Loss = 1.0526e-01, PNorm = 42.9766, GNorm = 1.0218, lr_0 = 7.2598e-04
Validation prc-auc = 0.383054
Epoch 195
Validation prc-auc = 0.354979
Epoch 196
Validation prc-auc = 0.341871
Epoch 197
Validation prc-auc = 0.328253
Epoch 198
Validation prc-auc = 0.330894
Epoch 199
Loss = 7.2458e-02, PNorm = 43.0364, GNorm = 0.4998, lr_0 = 7.1977e-04
Validation prc-auc = 0.331814
Epoch 200
Validation prc-auc = 0.338787
Epoch 201
Validation prc-auc = 0.351155
Epoch 202
Validation prc-auc = 0.368142
Epoch 203
Validation prc-auc = 0.353464
Epoch 204
Loss = 9.3861e-02, PNorm = 43.0900, GNorm = 1.3616, lr_0 = 7.1362e-04
Validation prc-auc = 0.336596
Epoch 205
Validation prc-auc = 0.322640
Epoch 206
Validation prc-auc = 0.324742
Epoch 207
Validation prc-auc = 0.319812
Epoch 208
Validation prc-auc = 0.317842
Epoch 209
Loss = 1.5967e-01, PNorm = 43.1401, GNorm = 1.4419, lr_0 = 7.0752e-04
Validation prc-auc = 0.323938
Epoch 210
Validation prc-auc = 0.327989
Epoch 211
Validation prc-auc = 0.338841
Epoch 212
Validation prc-auc = 0.360910
Epoch 213
Validation prc-auc = 0.358979
Epoch 214
Loss = 5.8609e-02, PNorm = 43.1904, GNorm = 0.9954, lr_0 = 7.0147e-04
Validation prc-auc = 0.340502
Epoch 215
Validation prc-auc = 0.368108
Epoch 216
Validation prc-auc = 0.361179
Epoch 217
Validation prc-auc = 0.352149
Epoch 218
Validation prc-auc = 0.351617
Epoch 219
Loss = 7.4385e-02, PNorm = 43.2415, GNorm = 1.2183, lr_0 = 6.9548e-04
Validation prc-auc = 0.347008
Epoch 220
Validation prc-auc = 0.346847
Epoch 221
Validation prc-auc = 0.348389
Epoch 222
Validation prc-auc = 0.349674
Epoch 223
Validation prc-auc = 0.363076
Epoch 224
Loss = 9.3349e-02, PNorm = 43.2809, GNorm = 0.5978, lr_0 = 6.8953e-04
Validation prc-auc = 0.339996
Epoch 225
Validation prc-auc = 0.327217
Epoch 226
Validation prc-auc = 0.325894
Epoch 227
Validation prc-auc = 0.309232
Epoch 228
Validation prc-auc = 0.308448
Epoch 229
Loss = 1.7262e-01, PNorm = 43.3282, GNorm = 2.2857, lr_0 = 6.8364e-04
Validation prc-auc = 0.308952
Epoch 230
Validation prc-auc = 0.309526
Epoch 231
Validation prc-auc = 0.316983
Epoch 232
Validation prc-auc = 0.326559
Epoch 233
Validation prc-auc = 0.325436
Epoch 234
Loss = 8.3176e-02, PNorm = 43.3744, GNorm = 1.9226, lr_0 = 6.7779e-04
Validation prc-auc = 0.319080
Epoch 235
Validation prc-auc = 0.319360
Epoch 236
Validation prc-auc = 0.321964
Epoch 237
Validation prc-auc = 0.327881
Epoch 238
Validation prc-auc = 0.332690
Epoch 239
Loss = 1.3170e-01, PNorm = 43.4092, GNorm = 1.5105, lr_0 = 6.7200e-04
Validation prc-auc = 0.338723
Epoch 240
Validation prc-auc = 0.333153
Epoch 241
Validation prc-auc = 0.324079
Epoch 242
Validation prc-auc = 0.317120
Epoch 243
Validation prc-auc = 0.320712
Epoch 244
Loss = 3.0776e-02, PNorm = 43.4499, GNorm = 0.7037, lr_0 = 6.6625e-04
Validation prc-auc = 0.315602
Epoch 245
Validation prc-auc = 0.315828
Epoch 246
Validation prc-auc = 0.317115
Epoch 247
Validation prc-auc = 0.315694
Epoch 248
Validation prc-auc = 0.317578
Epoch 249
Loss = 4.8133e-02, PNorm = 43.4900, GNorm = 0.4368, lr_0 = 6.6056e-04
Validation prc-auc = 0.316897
Epoch 250
Validation prc-auc = 0.317815
Epoch 251
Validation prc-auc = 0.325234
Epoch 252
Validation prc-auc = 0.329956
Epoch 253
Validation prc-auc = 0.321740
Epoch 254
Loss = 6.2309e-02, PNorm = 43.5247, GNorm = 1.1350, lr_0 = 6.5491e-04
Validation prc-auc = 0.314320
Epoch 255
Validation prc-auc = 0.313077
Epoch 256
Validation prc-auc = 0.309766
Epoch 257
Validation prc-auc = 0.308129
Epoch 258
Validation prc-auc = 0.334219
Epoch 259
Loss = 7.0462e-02, PNorm = 43.5609, GNorm = 0.9715, lr_0 = 6.4931e-04
Validation prc-auc = 0.336606
Epoch 260
Validation prc-auc = 0.341313
Epoch 261
Validation prc-auc = 0.352832
Epoch 262
Validation prc-auc = 0.343307
Epoch 263
Validation prc-auc = 0.342976
Epoch 264
Loss = 6.0855e-02, PNorm = 43.5967, GNorm = 2.0230, lr_0 = 6.4376e-04
Validation prc-auc = 0.344480
Epoch 265
Validation prc-auc = 0.333109
Epoch 266
Validation prc-auc = 0.332300
Epoch 267
Validation prc-auc = 0.331957
Epoch 268
Validation prc-auc = 0.328817
Epoch 269
Loss = 4.4360e-02, PNorm = 43.6395, GNorm = 0.5265, lr_0 = 6.3826e-04
Validation prc-auc = 0.357701
Epoch 270
Validation prc-auc = 0.369483
Epoch 271
Validation prc-auc = 0.371592
Epoch 272
Validation prc-auc = 0.373086
Epoch 273
Validation prc-auc = 0.356804
Epoch 274
Loss = 6.1650e-02, PNorm = 43.6751, GNorm = 0.3922, lr_0 = 6.3280e-04
Validation prc-auc = 0.356662
Epoch 275
Validation prc-auc = 0.348676
Epoch 276
Validation prc-auc = 0.348172
Epoch 277
Validation prc-auc = 0.357321
Epoch 278
Validation prc-auc = 0.372335
Epoch 279
Loss = 3.8108e-02, PNorm = 43.7081, GNorm = 0.9103, lr_0 = 6.2739e-04
Validation prc-auc = 0.382355
Epoch 280
Validation prc-auc = 0.383777
Epoch 281
Validation prc-auc = 0.367045
Epoch 282
Validation prc-auc = 0.356914
Epoch 283
Validation prc-auc = 0.347933
Epoch 284
Loss = 4.9138e-02, PNorm = 43.7442, GNorm = 1.3820, lr_0 = 6.2203e-04
Validation prc-auc = 0.335679
Epoch 285
Validation prc-auc = 0.333114
Epoch 286
Validation prc-auc = 0.306668
Epoch 287
Validation prc-auc = 0.338375
Epoch 288
Validation prc-auc = 0.336559
Epoch 289
Loss = 6.4508e-02, PNorm = 43.7741, GNorm = 1.6846, lr_0 = 6.1671e-04
Validation prc-auc = 0.326389
Epoch 290
Validation prc-auc = 0.324024
Epoch 291
Validation prc-auc = 0.287102
Epoch 292
Validation prc-auc = 0.279702
Epoch 293
Validation prc-auc = 0.273063
Epoch 294
Loss = 5.1875e-02, PNorm = 43.8020, GNorm = 1.3304, lr_0 = 6.1144e-04
Validation prc-auc = 0.295152
Epoch 295
Validation prc-auc = 0.300415
Epoch 296
Validation prc-auc = 0.326647
Epoch 297
Validation prc-auc = 0.344234
Epoch 298
Validation prc-auc = 0.368048
Epoch 299
Loss = 4.1757e-02, PNorm = 43.8356, GNorm = 0.8551, lr_0 = 6.0621e-04
Validation prc-auc = 0.376427
Model 0 best validation prc-auc = 0.425651 on epoch 162
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test prc-auc = 0.471997
Ensemble test prc-auc = 0.471997
Fold 2
Splitting data with seed 2
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=1728, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=300, out_features=300, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 873,601
Moving model to cuda
Epoch 0
Validation prc-auc = 0.098841
Epoch 1
Validation prc-auc = 0.261594
Epoch 2
Validation prc-auc = 0.392808
Epoch 3
Validation prc-auc = 0.424937
Epoch 4
Loss = 8.2195e-01, PNorm = 40.7407, GNorm = 2.9680, lr_0 = 6.5000e-04
Validation prc-auc = 0.420430
Epoch 5
Validation prc-auc = 0.383697
Epoch 6
Validation prc-auc = 0.377489
Epoch 7
Validation prc-auc = 0.383002
Epoch 8
Validation prc-auc = 0.379748
Epoch 9
Loss = 7.5633e-01, PNorm = 40.7919, GNorm = 3.3958, lr_0 = 9.9743e-04
Validation prc-auc = 0.372236
Epoch 10
Validation prc-auc = 0.357940
Epoch 11
Validation prc-auc = 0.347013
Epoch 12
Validation prc-auc = 0.357541
Epoch 13
Validation prc-auc = 0.373450
Epoch 14
Loss = 4.7821e-01, PNorm = 40.8528, GNorm = 1.8617, lr_0 = 9.8890e-04
Validation prc-auc = 0.388164
Epoch 15
Validation prc-auc = 0.404677
Epoch 16
Validation prc-auc = 0.431725
Epoch 17
Validation prc-auc = 0.432658
Epoch 18
Validation prc-auc = 0.415178
Epoch 19
Loss = 5.6619e-01, PNorm = 40.9080, GNorm = 2.0760, lr_0 = 9.8045e-04
Validation prc-auc = 0.402120
Epoch 20
Validation prc-auc = 0.376706
Epoch 21
Validation prc-auc = 0.360628
Epoch 22
Validation prc-auc = 0.346114
Epoch 23
Validation prc-auc = 0.340296
Epoch 24
Loss = 4.4704e-01, PNorm = 40.9553, GNorm = 1.6537, lr_0 = 9.7207e-04
Validation prc-auc = 0.340248
Epoch 25
Validation prc-auc = 0.342211
Epoch 26
Validation prc-auc = 0.341836
Epoch 27
Validation prc-auc = 0.341084
Epoch 28
Validation prc-auc = 0.333616
Epoch 29
Loss = 4.6859e-01, PNorm = 40.9976, GNorm = 2.3601, lr_0 = 9.6376e-04
Validation prc-auc = 0.336589
Epoch 30
Validation prc-auc = 0.361965
Epoch 31
Validation prc-auc = 0.376497
Epoch 32
Validation prc-auc = 0.376038
Epoch 33
Validation prc-auc = 0.373257
Epoch 34
Loss = 4.4228e-01, PNorm = 41.0401, GNorm = 1.6070, lr_0 = 9.5552e-04
Validation prc-auc = 0.369118
Epoch 35
Validation prc-auc = 0.365692
Epoch 36
Validation prc-auc = 0.360184
Epoch 37
Validation prc-auc = 0.349968
Epoch 38
Validation prc-auc = 0.349710
Epoch 39
Loss = 4.1071e-01, PNorm = 41.0861, GNorm = 2.3499, lr_0 = 9.4735e-04
Validation prc-auc = 0.347798
Epoch 40
Validation prc-auc = 0.342884
Epoch 41
Validation prc-auc = 0.342317
Epoch 42
Validation prc-auc = 0.342354
Epoch 43
Validation prc-auc = 0.344853
Epoch 44
Loss = 5.5096e-01, PNorm = 41.1379, GNorm = 5.3910, lr_0 = 9.3925e-04
Validation prc-auc = 0.349393
Epoch 45
Validation prc-auc = 0.353823
Epoch 46
Validation prc-auc = 0.352115
Epoch 47
Validation prc-auc = 0.352624
Epoch 48
Validation prc-auc = 0.348955
Epoch 49
Loss = 4.8252e-01, PNorm = 41.1930, GNorm = 2.0231, lr_0 = 9.3122e-04
Validation prc-auc = 0.348755
Epoch 50
Validation prc-auc = 0.348378
Epoch 51
Validation prc-auc = 0.350738
Epoch 52
Validation prc-auc = 0.359824
Epoch 53
Validation prc-auc = 0.370869
Epoch 54
Loss = 3.4310e-01, PNorm = 41.2474, GNorm = 1.5913, lr_0 = 9.2326e-04
Validation prc-auc = 0.378322
Epoch 55
Validation prc-auc = 0.376844
Epoch 56
Validation prc-auc = 0.372619
Epoch 57
Validation prc-auc = 0.377849
Epoch 58
Validation prc-auc = 0.369480
Epoch 59
Loss = 3.4613e-01, PNorm = 41.3017, GNorm = 2.0751, lr_0 = 9.1537e-04
Validation prc-auc = 0.328320
Epoch 60
Validation prc-auc = 0.322323
Epoch 61
Validation prc-auc = 0.354431
Epoch 62
Validation prc-auc = 0.299063
Epoch 63
Validation prc-auc = 0.288244
Epoch 64
Loss = 3.2871e-01, PNorm = 41.3582, GNorm = 2.0372, lr_0 = 9.0754e-04
Validation prc-auc = 0.285264
Epoch 65
Validation prc-auc = 0.281978
Epoch 66
Validation prc-auc = 0.276716
Epoch 67
Validation prc-auc = 0.275652
Epoch 68
Validation prc-auc = 0.276551
Epoch 69
Loss = 3.2906e-01, PNorm = 41.4215, GNorm = 1.3612, lr_0 = 8.9978e-04
Validation prc-auc = 0.276749
Epoch 70
Validation prc-auc = 0.281903
Epoch 71
Validation prc-auc = 0.266096
Epoch 72
Validation prc-auc = 0.285010
Epoch 73
Validation prc-auc = 0.356556
Epoch 74
Loss = 2.6750e-01, PNorm = 41.4874, GNorm = 2.5159, lr_0 = 8.9209e-04
Validation prc-auc = 0.372948
Epoch 75
Validation prc-auc = 0.381260
Epoch 76
Validation prc-auc = 0.382054
Epoch 77
Validation prc-auc = 0.386298
Epoch 78
Validation prc-auc = 0.387967
Epoch 79
Loss = 3.5395e-01, PNorm = 41.5500, GNorm = 1.5040, lr_0 = 8.8447e-04
Validation prc-auc = 0.362014
Epoch 80
Validation prc-auc = 0.305044
Epoch 81
Validation prc-auc = 0.287451
Epoch 82
Validation prc-auc = 0.316674
Epoch 83
Validation prc-auc = 0.313564
Epoch 84
Loss = 3.1346e-01, PNorm = 41.6151, GNorm = 1.2010, lr_0 = 8.7691e-04
Validation prc-auc = 0.310667
Epoch 85
Validation prc-auc = 0.290732
Epoch 86
Validation prc-auc = 0.302098
Epoch 87
Validation prc-auc = 0.315983
Epoch 88
Validation prc-auc = 0.345882
Epoch 89
Loss = 2.5637e-01, PNorm = 41.6845, GNorm = 1.4510, lr_0 = 8.6941e-04
Validation prc-auc = 0.342871
Epoch 90
Validation prc-auc = 0.343781
Epoch 91
Validation prc-auc = 0.342518
Epoch 92
Validation prc-auc = 0.312711
Epoch 93
Validation prc-auc = 0.310275
Epoch 94
Loss = 1.8863e-01, PNorm = 41.7539, GNorm = 0.9993, lr_0 = 8.6198e-04
Validation prc-auc = 0.307825
Epoch 95
Validation prc-auc = 0.308339
Epoch 96
Validation prc-auc = 0.321095
Epoch 97
Validation prc-auc = 0.321393
Epoch 98
Validation prc-auc = 0.322238
Epoch 99
Loss = 1.6067e-01, PNorm = 41.8221, GNorm = 1.2421, lr_0 = 8.5461e-04
Validation prc-auc = 0.323598
Epoch 100
Validation prc-auc = 0.303554
Epoch 101
Validation prc-auc = 0.301304
Epoch 102
Validation prc-auc = 0.345362
Epoch 103
Validation prc-auc = 0.344386
Epoch 104
Loss = 2.2863e-01, PNorm = 41.8864, GNorm = 1.5270, lr_0 = 8.4730e-04
Validation prc-auc = 0.364806
Epoch 105
Validation prc-auc = 0.362963
Epoch 106
Validation prc-auc = 0.369260
Epoch 107
Validation prc-auc = 0.362526
Epoch 108
Validation prc-auc = 0.350876
Epoch 109
Loss = 1.8273e-01, PNorm = 41.9511, GNorm = 1.9160, lr_0 = 8.4006e-04
Validation prc-auc = 0.344098
Epoch 110
Validation prc-auc = 0.326123
Epoch 111
Validation prc-auc = 0.309611
Epoch 112
Validation prc-auc = 0.302159
Epoch 113
Validation prc-auc = 0.319484
Epoch 114
Loss = 2.4323e-01, PNorm = 42.0154, GNorm = 1.7587, lr_0 = 8.3288e-04
Validation prc-auc = 0.319273
Epoch 115
Validation prc-auc = 0.302100
Epoch 116
Validation prc-auc = 0.350444
Epoch 117
Validation prc-auc = 0.371225
Epoch 118
Validation prc-auc = 0.378453
Epoch 119
Loss = 1.4146e-01, PNorm = 42.0837, GNorm = 1.3570, lr_0 = 8.2576e-04
Validation prc-auc = 0.379727
Epoch 120
Validation prc-auc = 0.364975
Epoch 121
Validation prc-auc = 0.340275
Epoch 122
Validation prc-auc = 0.348722
Epoch 123
Validation prc-auc = 0.346548
Epoch 124
Loss = 1.6632e-01, PNorm = 42.1513, GNorm = 1.1822, lr_0 = 8.1870e-04
Validation prc-auc = 0.334461
Epoch 125
Validation prc-auc = 0.318997
Epoch 126
Validation prc-auc = 0.308583
Epoch 127
Validation prc-auc = 0.314183
Epoch 128
Validation prc-auc = 0.341337
Epoch 129
Loss = 1.7461e-01, PNorm = 42.2213, GNorm = 1.3920, lr_0 = 8.1170e-04
Validation prc-auc = 0.341633
Epoch 130
Validation prc-auc = 0.341900
Epoch 131
Validation prc-auc = 0.332587
Epoch 132
Validation prc-auc = 0.345926
Epoch 133
Validation prc-auc = 0.368226
Epoch 134
Loss = 1.4073e-01, PNorm = 42.2985, GNorm = 2.0450, lr_0 = 8.0476e-04
Validation prc-auc = 0.370511
Epoch 135
Validation prc-auc = 0.380194
Epoch 136
Validation prc-auc = 0.382662
Epoch 137
Validation prc-auc = 0.376946
Epoch 138
Validation prc-auc = 0.378729
Epoch 139
Loss = 1.7870e-01, PNorm = 42.3696, GNorm = 1.0942, lr_0 = 7.9788e-04
Validation prc-auc = 0.382805
Epoch 140
Validation prc-auc = 0.383938
Epoch 141
Validation prc-auc = 0.363244
Epoch 142
Validation prc-auc = 0.303617
Epoch 143
Validation prc-auc = 0.314448
Epoch 144
Loss = 2.3072e-01, PNorm = 42.4252, GNorm = 1.5489, lr_0 = 7.9106e-04
Validation prc-auc = 0.314670
Epoch 145
Validation prc-auc = 0.298047
Epoch 146
Validation prc-auc = 0.285829
Epoch 147
Validation prc-auc = 0.274821
Epoch 148
Validation prc-auc = 0.282465
Epoch 149
Loss = 2.6658e-01, PNorm = 42.4933, GNorm = 2.9652, lr_0 = 7.8430e-04
Validation prc-auc = 0.292452
Epoch 150
Validation prc-auc = 0.330374
Epoch 151
Validation prc-auc = 0.415620
Epoch 152
Validation prc-auc = 0.418505
Epoch 153
Validation prc-auc = 0.431858
Epoch 154
Loss = 1.7071e-01, PNorm = 42.5589, GNorm = 1.1532, lr_0 = 7.7759e-04
Validation prc-auc = 0.406508
Epoch 155
Validation prc-auc = 0.377378
Epoch 156
Validation prc-auc = 0.361767
Epoch 157
Validation prc-auc = 0.334482
Epoch 158
Validation prc-auc = 0.331262
Epoch 159
Loss = 1.5611e-01, PNorm = 42.6208, GNorm = 0.9131, lr_0 = 7.7095e-04
Validation prc-auc = 0.335094
Epoch 160
Validation prc-auc = 0.319172
Epoch 161
Validation prc-auc = 0.355084
Epoch 162
Validation prc-auc = 0.364812
Epoch 163
Validation prc-auc = 0.371682
Epoch 164
Loss = 1.5189e-01, PNorm = 42.6791, GNorm = 1.8837, lr_0 = 7.6436e-04
Validation prc-auc = 0.357922
Epoch 165
Validation prc-auc = 0.354376
Epoch 166
Validation prc-auc = 0.326488
Epoch 167
Validation prc-auc = 0.321640
Epoch 168
Validation prc-auc = 0.319842
Epoch 169
Loss = 1.2301e-01, PNorm = 42.7338, GNorm = 1.0252, lr_0 = 7.5782e-04
Validation prc-auc = 0.314925
Epoch 170
Validation prc-auc = 0.293631
Epoch 171
Validation prc-auc = 0.290163
Epoch 172
Validation prc-auc = 0.298592
Epoch 173
Validation prc-auc = 0.311234
Epoch 174
Loss = 1.7671e-01, PNorm = 42.7818, GNorm = 0.7343, lr_0 = 7.5134e-04
Validation prc-auc = 0.315962
Epoch 175
Validation prc-auc = 0.315689
Epoch 176
Validation prc-auc = 0.319731
Epoch 177
Validation prc-auc = 0.307020
Epoch 178
Validation prc-auc = 0.303053
Epoch 179
Loss = 1.4686e-01, PNorm = 42.8301, GNorm = 1.0275, lr_0 = 7.4492e-04
Validation prc-auc = 0.291496
Epoch 180
Validation prc-auc = 0.284772
Epoch 181
Validation prc-auc = 0.279678
Epoch 182
Validation prc-auc = 0.283423
Epoch 183
Validation prc-auc = 0.307206
Epoch 184
Loss = 1.0043e-01, PNorm = 42.8802, GNorm = 1.1348, lr_0 = 7.3855e-04
Validation prc-auc = 0.318944
Epoch 185
Validation prc-auc = 0.314441
Epoch 186
Validation prc-auc = 0.325160
Epoch 187
Validation prc-auc = 0.363267
Epoch 188
Validation prc-auc = 0.318910
Epoch 189
Loss = 1.5702e-01, PNorm = 42.9251, GNorm = 1.7582, lr_0 = 7.3224e-04
Validation prc-auc = 0.305146
Epoch 190
Validation prc-auc = 0.296699
Epoch 191
Validation prc-auc = 0.281498
Epoch 192
Validation prc-auc = 0.279351
Epoch 193
Validation prc-auc = 0.292072
Epoch 194
Loss = 9.4178e-02, PNorm = 42.9753, GNorm = 1.0374, lr_0 = 7.2598e-04
Validation prc-auc = 0.310229
Epoch 195
Validation prc-auc = 0.310848
Epoch 196
Validation prc-auc = 0.308669
Epoch 197
Validation prc-auc = 0.297131
Epoch 198
Validation prc-auc = 0.283409
Epoch 199
Loss = 1.5155e-01, PNorm = 43.0267, GNorm = 1.7037, lr_0 = 7.1977e-04
Validation prc-auc = 0.279707
Epoch 200
Validation prc-auc = 0.288995
Epoch 201
Validation prc-auc = 0.297837
Epoch 202
Validation prc-auc = 0.301678
Epoch 203
Validation prc-auc = 0.309451
Epoch 204
Loss = 3.3910e-02, PNorm = 43.0751, GNorm = 0.5593, lr_0 = 7.1362e-04
Validation prc-auc = 0.321362
Epoch 205
Validation prc-auc = 0.320640
Epoch 206
Validation prc-auc = 0.308942
Epoch 207
Validation prc-auc = 0.296126
Epoch 208
Validation prc-auc = 0.288968
Epoch 209
Loss = 6.0572e-02, PNorm = 43.1139, GNorm = 1.0328, lr_0 = 7.0752e-04
Validation prc-auc = 0.295171
Epoch 210
Validation prc-auc = 0.305227
Epoch 211
Validation prc-auc = 0.300677
Epoch 212
Validation prc-auc = 0.304216
Epoch 213
Validation prc-auc = 0.295135
Epoch 214
Loss = 9.2966e-02, PNorm = 43.1588, GNorm = 1.6773, lr_0 = 7.0147e-04
Validation prc-auc = 0.290970
Epoch 215
Validation prc-auc = 0.263495
Epoch 216
Validation prc-auc = 0.294678
Epoch 217
Validation prc-auc = 0.303874
Epoch 218
Validation prc-auc = 0.315064
Epoch 219
Loss = 1.0657e-01, PNorm = 43.2040, GNorm = 1.2006, lr_0 = 6.9548e-04
Validation prc-auc = 0.321323
Epoch 220
Validation prc-auc = 0.314018
Epoch 221
Validation prc-auc = 0.309705
Epoch 222
Validation prc-auc = 0.302973
Epoch 223
Validation prc-auc = 0.293941
Epoch 224
Loss = 8.4645e-02, PNorm = 43.2515, GNorm = 1.6083, lr_0 = 6.8953e-04
Validation prc-auc = 0.286048
Epoch 225
Validation prc-auc = 0.289285
Epoch 226
Validation prc-auc = 0.293560
Epoch 227
Validation prc-auc = 0.313506
Epoch 228
Validation prc-auc = 0.290246
Epoch 229
Loss = 1.3837e-01, PNorm = 43.2932, GNorm = 2.0255, lr_0 = 6.8364e-04
Validation prc-auc = 0.282998
Epoch 230
Validation prc-auc = 0.294502
Epoch 231
Validation prc-auc = 0.315859
Epoch 232
Validation prc-auc = 0.303389
Epoch 233
Validation prc-auc = 0.306261
Epoch 234
Loss = 7.6661e-02, PNorm = 43.3472, GNorm = 1.0170, lr_0 = 6.7779e-04
Validation prc-auc = 0.308875
Epoch 235
Validation prc-auc = 0.315956
Epoch 236
Validation prc-auc = 0.329628
Epoch 237
Validation prc-auc = 0.334694
Epoch 238
Validation prc-auc = 0.337397
Epoch 239
Loss = 7.8658e-02, PNorm = 43.3963, GNorm = 1.2141, lr_0 = 6.7200e-04
Validation prc-auc = 0.334566
Epoch 240
Validation prc-auc = 0.322582
Epoch 241
Validation prc-auc = 0.304398
Epoch 242
Validation prc-auc = 0.294670
Epoch 243
Validation prc-auc = 0.284643
Epoch 244
Loss = 7.2744e-02, PNorm = 43.4375, GNorm = 1.0798, lr_0 = 6.6625e-04
Validation prc-auc = 0.282584
Epoch 245
Validation prc-auc = 0.281384
Epoch 246
Validation prc-auc = 0.279484
Epoch 247
Validation prc-auc = 0.287583
Epoch 248
Validation prc-auc = 0.296461
Epoch 249
Loss = 6.6399e-02, PNorm = 43.4765, GNorm = 1.4303, lr_0 = 6.6056e-04
Validation prc-auc = 0.301170
Epoch 250
Validation prc-auc = 0.304208
Epoch 251
Validation prc-auc = 0.305712
Epoch 252
Validation prc-auc = 0.304401
Epoch 253
Validation prc-auc = 0.304888
Epoch 254
Loss = 2.4172e-02, PNorm = 43.5096, GNorm = 0.4242, lr_0 = 6.5491e-04
Validation prc-auc = 0.298607
Epoch 255
Validation prc-auc = 0.300384
Epoch 256
Validation prc-auc = 0.298853
Epoch 257
Validation prc-auc = 0.300460
Epoch 258
Validation prc-auc = 0.294422
Epoch 259
Loss = 3.3896e-02, PNorm = 43.5383, GNorm = 0.6496, lr_0 = 6.4931e-04
Validation prc-auc = 0.298085
Epoch 260
Validation prc-auc = 0.306359
Epoch 261
Validation prc-auc = 0.309106
Epoch 262
Validation prc-auc = 0.307600
Epoch 263
Validation prc-auc = 0.308232
Epoch 264
Loss = 6.2309e-02, PNorm = 43.5671, GNorm = 0.7753, lr_0 = 6.4376e-04
Validation prc-auc = 0.301506
Epoch 265
Validation prc-auc = 0.306945
Epoch 266
Validation prc-auc = 0.311975
Epoch 267
Validation prc-auc = 0.315414
Epoch 268
Validation prc-auc = 0.322606
Epoch 269
Loss = 9.8973e-02, PNorm = 43.6052, GNorm = 0.8412, lr_0 = 6.3826e-04
Validation prc-auc = 0.322324
Epoch 270
Validation prc-auc = 0.320024
Epoch 271
Validation prc-auc = 0.290624
Epoch 272
Validation prc-auc = 0.288938
Epoch 273
Validation prc-auc = 0.295035
Epoch 274
Loss = 2.6231e-02, PNorm = 43.6532, GNorm = 1.4424, lr_0 = 6.3280e-04
Validation prc-auc = 0.328540
Epoch 275
Validation prc-auc = 0.332626
Epoch 276
Validation prc-auc = 0.341450
Epoch 277
Validation prc-auc = 0.336690
Epoch 278
Validation prc-auc = 0.332255
Epoch 279
Loss = 6.4043e-02, PNorm = 43.6988, GNorm = 1.4230, lr_0 = 6.2739e-04
Validation prc-auc = 0.332028
Epoch 280
Validation prc-auc = 0.339132
Epoch 281
Validation prc-auc = 0.325788
Epoch 282
Validation prc-auc = 0.316624
Epoch 283
Validation prc-auc = 0.357496
Epoch 284
Loss = 7.4414e-02, PNorm = 43.7413, GNorm = 0.7465, lr_0 = 6.2203e-04
Validation prc-auc = 0.341922
Epoch 285
Validation prc-auc = 0.332815
Epoch 286
Validation prc-auc = 0.334270
Epoch 287
Validation prc-auc = 0.347251
Epoch 288
Validation prc-auc = 0.347806
Epoch 289
Loss = 7.2393e-02, PNorm = 43.7828, GNorm = 0.3633, lr_0 = 6.1671e-04
Validation prc-auc = 0.354057
Epoch 290
Validation prc-auc = 0.364696
Epoch 291
Validation prc-auc = 0.367164
Epoch 292
Validation prc-auc = 0.362425
Epoch 293
Validation prc-auc = 0.348677
Epoch 294
Loss = 1.0539e-01, PNorm = 43.8174, GNorm = 0.6536, lr_0 = 6.1144e-04
Validation prc-auc = 0.323653
Epoch 295
Validation prc-auc = 0.312418
Epoch 296
Validation prc-auc = 0.311919
Epoch 297
Validation prc-auc = 0.310167
Epoch 298
Validation prc-auc = 0.314397
Epoch 299
Loss = 4.8360e-02, PNorm = 43.8467, GNorm = 0.6065, lr_0 = 6.0621e-04
Validation prc-auc = 0.313870
Model 0 best validation prc-auc = 0.432658 on epoch 17
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test prc-auc = 0.378929
Ensemble test prc-auc = 0.378929
Fold 3
Splitting data with seed 3
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=1728, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=300, out_features=300, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 873,601
Moving model to cuda
Epoch 0
Validation prc-auc = 0.098468
Epoch 1
Validation prc-auc = 0.203054
Epoch 2
Validation prc-auc = 0.330382
Epoch 3
Validation prc-auc = 0.334849
Epoch 4
Loss = 6.3285e-01, PNorm = 40.7422, GNorm = 2.7560, lr_0 = 6.5000e-04
Validation prc-auc = 0.355468
Epoch 5
Validation prc-auc = 0.349656
Epoch 6
Validation prc-auc = 0.347477
Epoch 7
Validation prc-auc = 0.358563
Epoch 8
Validation prc-auc = 0.363782
Epoch 9
Loss = 5.8552e-01, PNorm = 40.8016, GNorm = 3.0865, lr_0 = 9.9743e-04
Validation prc-auc = 0.331124
Epoch 10
Validation prc-auc = 0.319695
Epoch 11
Validation prc-auc = 0.305796
Epoch 12
Validation prc-auc = 0.298347
Epoch 13
Validation prc-auc = 0.297509
Epoch 14
Loss = 6.2437e-01, PNorm = 40.8679, GNorm = 6.1536, lr_0 = 9.8890e-04
Validation prc-auc = 0.316269
Epoch 15
Validation prc-auc = 0.332714
Epoch 16
Validation prc-auc = 0.388295
Epoch 17
Validation prc-auc = 0.400637
Epoch 18
Validation prc-auc = 0.407566
Epoch 19
Loss = 5.9211e-01, PNorm = 40.9245, GNorm = 2.4236, lr_0 = 9.8045e-04
Validation prc-auc = 0.404109
Epoch 20
Validation prc-auc = 0.410019
Epoch 21
Validation prc-auc = 0.414565
Epoch 22
Validation prc-auc = 0.399571
Epoch 23
Validation prc-auc = 0.378964
Epoch 24
Loss = 5.2916e-01, PNorm = 40.9718, GNorm = 1.8779, lr_0 = 9.7207e-04
Validation prc-auc = 0.371030
Epoch 25
Validation prc-auc = 0.366346
Epoch 26
Validation prc-auc = 0.365703
Epoch 27
Validation prc-auc = 0.361897
Epoch 28
Validation prc-auc = 0.362231
Epoch 29
Loss = 5.3110e-01, PNorm = 41.0129, GNorm = 2.2426, lr_0 = 9.6376e-04
Validation prc-auc = 0.363798
Epoch 30
Validation prc-auc = 0.369948
Epoch 31
Validation prc-auc = 0.372698
Epoch 32
Validation prc-auc = 0.364285
Epoch 33
Validation prc-auc = 0.358116
Epoch 34
Loss = 4.2983e-01, PNorm = 41.0575, GNorm = 1.7719, lr_0 = 9.5552e-04
Validation prc-auc = 0.354232
Epoch 35
Validation prc-auc = 0.355387
Epoch 36
Validation prc-auc = 0.350336
Epoch 37
Validation prc-auc = 0.349427
Epoch 38
Validation prc-auc = 0.348204
Epoch 39
Loss = 4.0520e-01, PNorm = 41.1067, GNorm = 1.5006, lr_0 = 9.4735e-04
Validation prc-auc = 0.360423
Epoch 40
Validation prc-auc = 0.371570
Epoch 41
Validation prc-auc = 0.358598
Epoch 42
Validation prc-auc = 0.341881
Epoch 43
Validation prc-auc = 0.341351
Epoch 44
Loss = 3.9093e-01, PNorm = 41.1533, GNorm = 1.5573, lr_0 = 9.3925e-04
Validation prc-auc = 0.322769
Epoch 45
Validation prc-auc = 0.323022
Epoch 46
Validation prc-auc = 0.310635
Epoch 47
Validation prc-auc = 0.313047
Epoch 48
Validation prc-auc = 0.319157
Epoch 49
Loss = 3.5235e-01, PNorm = 41.2018, GNorm = 1.6585, lr_0 = 9.3122e-04
Validation prc-auc = 0.316416
Epoch 50
Validation prc-auc = 0.309082
Epoch 51
Validation prc-auc = 0.312872
Epoch 52
Validation prc-auc = 0.316811
Epoch 53
Validation prc-auc = 0.326160
Epoch 54
Loss = 3.8460e-01, PNorm = 41.2522, GNorm = 1.7264, lr_0 = 9.2326e-04
Validation prc-auc = 0.321022
Epoch 55
Validation prc-auc = 0.316444
Epoch 56
Validation prc-auc = 0.321589
Epoch 57
Validation prc-auc = 0.321955
Epoch 58
Validation prc-auc = 0.321656
Epoch 59
Loss = 3.5136e-01, PNorm = 41.3059, GNorm = 1.5275, lr_0 = 9.1537e-04
Validation prc-auc = 0.309815
Epoch 60
Validation prc-auc = 0.309247
Epoch 61
Validation prc-auc = 0.314745
Epoch 62
Validation prc-auc = 0.309263
Epoch 63
Validation prc-auc = 0.314022
Epoch 64
Loss = 2.5285e-01, PNorm = 41.3609, GNorm = 1.2983, lr_0 = 9.0754e-04
Validation prc-auc = 0.322208
Epoch 65
Validation prc-auc = 0.348942
Epoch 66
Validation prc-auc = 0.373771
Epoch 67
Validation prc-auc = 0.397616
Epoch 68
Validation prc-auc = 0.401575
Epoch 69
Loss = 3.4091e-01, PNorm = 41.4219, GNorm = 1.6964, lr_0 = 8.9978e-04
Validation prc-auc = 0.396667
Epoch 70
Validation prc-auc = 0.374513
Epoch 71
Validation prc-auc = 0.378101
Epoch 72
Validation prc-auc = 0.379953
Epoch 73
Validation prc-auc = 0.366848
Epoch 74
Loss = 2.5965e-01, PNorm = 41.4855, GNorm = 1.7330, lr_0 = 8.9209e-04
Validation prc-auc = 0.338877
Epoch 75
Validation prc-auc = 0.334548
Epoch 76
Validation prc-auc = 0.338730
Epoch 77
Validation prc-auc = 0.332786
Epoch 78
Validation prc-auc = 0.327998
Epoch 79
Loss = 2.6787e-01, PNorm = 41.5465, GNorm = 2.3429, lr_0 = 8.8447e-04
Validation prc-auc = 0.330583
Epoch 80
Validation prc-auc = 0.333952
Epoch 81
Validation prc-auc = 0.336706
Epoch 82
Validation prc-auc = 0.376575
Epoch 83
Validation prc-auc = 0.392814
Epoch 84
Loss = 2.6104e-01, PNorm = 41.6148, GNorm = 1.8248, lr_0 = 8.7691e-04
Validation prc-auc = 0.401339
Epoch 85
Validation prc-auc = 0.397700
Epoch 86
Validation prc-auc = 0.387933
Epoch 87
Validation prc-auc = 0.392595
Epoch 88
Validation prc-auc = 0.396691
Epoch 89
Loss = 3.8675e-01, PNorm = 41.6816, GNorm = 1.8142, lr_0 = 8.6941e-04
Validation prc-auc = 0.389260
Epoch 90
Validation prc-auc = 0.367494
Epoch 91
Validation prc-auc = 0.339323
Epoch 92
Validation prc-auc = 0.346819
Epoch 93
Validation prc-auc = 0.352780
Epoch 94
Loss = 2.4093e-01, PNorm = 41.7403, GNorm = 1.1116, lr_0 = 8.6198e-04
Validation prc-auc = 0.372397
Epoch 95
Validation prc-auc = 0.369399
Epoch 96
Validation prc-auc = 0.403785
Epoch 97
Validation prc-auc = 0.423829
Epoch 98
Validation prc-auc = 0.404595
Epoch 99
Loss = 1.7431e-01, PNorm = 41.8038, GNorm = 1.2881, lr_0 = 8.5461e-04
Validation prc-auc = 0.402738
Epoch 100
Validation prc-auc = 0.387874
Epoch 101
Validation prc-auc = 0.374936
Epoch 102
Validation prc-auc = 0.336322
Epoch 103
Validation prc-auc = 0.330601
Epoch 104
Loss = 1.6696e-01, PNorm = 41.8595, GNorm = 1.1900, lr_0 = 8.4730e-04
Validation prc-auc = 0.318099
Epoch 105
Validation prc-auc = 0.310718
Epoch 106
Validation prc-auc = 0.300507
Epoch 107
Validation prc-auc = 0.308736
Epoch 108
Validation prc-auc = 0.315341
Epoch 109
Loss = 2.5648e-01, PNorm = 41.9243, GNorm = 2.4213, lr_0 = 8.4006e-04
Validation prc-auc = 0.328336
Epoch 110
Validation prc-auc = 0.331787
Epoch 111
Validation prc-auc = 0.337424
Epoch 112
Validation prc-auc = 0.327763
Epoch 113
Validation prc-auc = 0.304604
Epoch 114
Loss = 2.8020e-01, PNorm = 41.9943, GNorm = 2.1297, lr_0 = 8.3288e-04
Validation prc-auc = 0.303548
Epoch 115
Validation prc-auc = 0.341117
Epoch 116
Validation prc-auc = 0.352643
Epoch 117
Validation prc-auc = 0.364818
Epoch 118
Validation prc-auc = 0.345604
Epoch 119
Loss = 2.1140e-01, PNorm = 42.0597, GNorm = 1.3714, lr_0 = 8.2576e-04
Validation prc-auc = 0.305381
Epoch 120
Validation prc-auc = 0.305884
Epoch 121
Validation prc-auc = 0.314016
Epoch 122
Validation prc-auc = 0.332903
Epoch 123
Validation prc-auc = 0.338279
Epoch 124
Loss = 1.9738e-01, PNorm = 42.1297, GNorm = 1.0570, lr_0 = 8.1870e-04
Validation prc-auc = 0.331329
Epoch 125
Validation prc-auc = 0.323528
Epoch 126
Validation prc-auc = 0.310819
Epoch 127
Validation prc-auc = 0.305918
Epoch 128
Validation prc-auc = 0.300960
Epoch 129
Loss = 2.2541e-01, PNorm = 42.1973, GNorm = 1.2451, lr_0 = 8.1170e-04
Validation prc-auc = 0.309222
Epoch 130
Validation prc-auc = 0.327911
Epoch 131
Validation prc-auc = 0.339101
Epoch 132
Validation prc-auc = 0.309878
Epoch 133
Validation prc-auc = 0.288757
Epoch 134
Loss = 2.6711e-01, PNorm = 42.2534, GNorm = 1.8159, lr_0 = 8.0476e-04
Validation prc-auc = 0.288885
Epoch 135
Validation prc-auc = 0.298188
Epoch 136
Validation prc-auc = 0.295057
Epoch 137
Validation prc-auc = 0.292220
Epoch 138
Validation prc-auc = 0.306488
Epoch 139
Loss = 2.5168e-01, PNorm = 42.3225, GNorm = 2.0451, lr_0 = 7.9788e-04
Validation prc-auc = 0.306354
Epoch 140
Validation prc-auc = 0.317512
Epoch 141
Validation prc-auc = 0.330994
Epoch 142
Validation prc-auc = 0.347354
Epoch 143
Validation prc-auc = 0.360418
Epoch 144
Loss = 2.0323e-01, PNorm = 42.3876, GNorm = 1.2892, lr_0 = 7.9106e-04
Validation prc-auc = 0.364658
Epoch 145
Validation prc-auc = 0.354720
Epoch 146
Validation prc-auc = 0.338085
Epoch 147
Validation prc-auc = 0.326718
Epoch 148
Validation prc-auc = 0.323382
Epoch 149
Loss = 1.3298e-01, PNorm = 42.4441, GNorm = 1.1455, lr_0 = 7.8430e-04
Validation prc-auc = 0.321437
Epoch 150
Validation prc-auc = 0.316697
Epoch 151
Validation prc-auc = 0.315471
Epoch 152
Validation prc-auc = 0.301105
Epoch 153
Validation prc-auc = 0.296271
Epoch 154
Loss = 8.0767e-02, PNorm = 42.5042, GNorm = 1.1325, lr_0 = 7.7759e-04
Validation prc-auc = 0.305364
Epoch 155
Validation prc-auc = 0.302714
Epoch 156
Validation prc-auc = 0.315837
Epoch 157
Validation prc-auc = 0.329921
Epoch 158
Validation prc-auc = 0.316611
Epoch 159
Loss = 1.4896e-01, PNorm = 42.5519, GNorm = 1.4225, lr_0 = 7.7095e-04
Validation prc-auc = 0.288694
Epoch 160
Validation prc-auc = 0.286469
Epoch 161
Validation prc-auc = 0.289641
Epoch 162
Validation prc-auc = 0.295637
Epoch 163
Validation prc-auc = 0.306088
Epoch 164
Loss = 1.0363e-01, PNorm = 42.6199, GNorm = 0.8694, lr_0 = 7.6436e-04
Validation prc-auc = 0.318526
Epoch 165
Validation prc-auc = 0.326653
Epoch 166
Validation prc-auc = 0.315896
Epoch 167
Validation prc-auc = 0.311134
Epoch 168
Validation prc-auc = 0.310877
Epoch 169
Loss = 6.8914e-02, PNorm = 42.6766, GNorm = 0.8465, lr_0 = 7.5782e-04
Validation prc-auc = 0.326718
Epoch 170
Validation prc-auc = 0.337134
Epoch 171
Validation prc-auc = 0.341364
Epoch 172
Validation prc-auc = 0.339171
Epoch 173
Validation prc-auc = 0.328521
Epoch 174
Loss = 1.2268e-01, PNorm = 42.7273, GNorm = 1.9406, lr_0 = 7.5134e-04
Validation prc-auc = 0.319471
Epoch 175
Validation prc-auc = 0.313503
Epoch 176
Validation prc-auc = 0.326997
Epoch 177
Validation prc-auc = 0.343502
Epoch 178
Validation prc-auc = 0.340063
Epoch 179
Loss = 1.5927e-01, PNorm = 42.7787, GNorm = 1.6344, lr_0 = 7.4492e-04
Validation prc-auc = 0.338206
Epoch 180
Validation prc-auc = 0.316059
Epoch 181
Validation prc-auc = 0.293762
Epoch 182
Validation prc-auc = 0.290494
Epoch 183
Validation prc-auc = 0.290967
Epoch 184
Loss = 1.6874e-01, PNorm = 42.8321, GNorm = 1.2029, lr_0 = 7.3855e-04
Validation prc-auc = 0.304109
Epoch 185
Validation prc-auc = 0.320972
Epoch 186
Validation prc-auc = 0.332790
Epoch 187
Validation prc-auc = 0.332925
Epoch 188
Validation prc-auc = 0.334005
Epoch 189
Loss = 1.4430e-01, PNorm = 42.8893, GNorm = 1.5306, lr_0 = 7.3224e-04
Validation prc-auc = 0.323632
Epoch 190
Validation prc-auc = 0.316281
Epoch 191
Validation prc-auc = 0.318653
Epoch 192
Validation prc-auc = 0.324767
Epoch 193
Validation prc-auc = 0.331934
Epoch 194
Loss = 9.8855e-02, PNorm = 42.9389, GNorm = 1.5922, lr_0 = 7.2598e-04
Validation prc-auc = 0.333853
Epoch 195
Validation prc-auc = 0.338272
Epoch 196
Validation prc-auc = 0.321730
Epoch 197
Validation prc-auc = 0.311214
Epoch 198
Validation prc-auc = 0.307455
Epoch 199
Loss = 1.2961e-01, PNorm = 42.9943, GNorm = 1.4505, lr_0 = 7.1977e-04
Validation prc-auc = 0.313418
Epoch 200
Validation prc-auc = 0.330703
Epoch 201
Validation prc-auc = 0.336639
Epoch 202
Validation prc-auc = 0.336462
Epoch 203
Validation prc-auc = 0.327841
Epoch 204
Loss = 9.8566e-02, PNorm = 43.0515, GNorm = 1.1410, lr_0 = 7.1362e-04
Validation prc-auc = 0.323953
Epoch 205
Validation prc-auc = 0.324456
Epoch 206
Validation prc-auc = 0.322637
Epoch 207
Validation prc-auc = 0.321882
Epoch 208
Validation prc-auc = 0.298376
Epoch 209
Loss = 1.4039e-01, PNorm = 43.1011, GNorm = 1.3167, lr_0 = 7.0752e-04
Validation prc-auc = 0.301183
Epoch 210
Validation prc-auc = 0.304885
Epoch 211
Validation prc-auc = 0.293722
Epoch 212
Validation prc-auc = 0.308281
Epoch 213
Validation prc-auc = 0.295257
Epoch 214
Loss = 1.3121e-01, PNorm = 43.1484, GNorm = 1.5533, lr_0 = 7.0147e-04
Validation prc-auc = 0.295398
Epoch 215
Validation prc-auc = 0.304840
Epoch 216
Validation prc-auc = 0.320318
Epoch 217
Validation prc-auc = 0.321912
Epoch 218
Validation prc-auc = 0.304371
Epoch 219
Loss = 7.1626e-02, PNorm = 43.1939, GNorm = 0.9874, lr_0 = 6.9548e-04
Validation prc-auc = 0.298352
Epoch 220
Validation prc-auc = 0.292652
Epoch 221
Validation prc-auc = 0.293192
Epoch 222
Validation prc-auc = 0.299085
Epoch 223
Validation prc-auc = 0.310664
Epoch 224
Loss = 5.2652e-02, PNorm = 43.2449, GNorm = 0.8433, lr_0 = 6.8953e-04
Validation prc-auc = 0.326963
Epoch 225
Validation prc-auc = 0.351176
Epoch 226
Validation prc-auc = 0.361475
Epoch 227
Validation prc-auc = 0.352446
Epoch 228
Validation prc-auc = 0.335624
Epoch 229
Loss = 8.4589e-02, PNorm = 43.2928, GNorm = 0.7054, lr_0 = 6.8364e-04
Validation prc-auc = 0.322010
Epoch 230
Validation prc-auc = 0.321122
Epoch 231
Validation prc-auc = 0.314015
Epoch 232
Validation prc-auc = 0.315608
Epoch 233
Validation prc-auc = 0.312263
Epoch 234
Loss = 7.9557e-02, PNorm = 43.3365, GNorm = 1.2961, lr_0 = 6.7779e-04
Validation prc-auc = 0.316875
Epoch 235
Validation prc-auc = 0.317095
Epoch 236
Validation prc-auc = 0.325005
Epoch 237
Validation prc-auc = 0.328679
Epoch 238
Validation prc-auc = 0.311920
Epoch 239
Loss = 7.0563e-02, PNorm = 43.3800, GNorm = 1.0414, lr_0 = 6.7200e-04
Validation prc-auc = 0.304972
Epoch 240
Validation prc-auc = 0.298168
Epoch 241
Validation prc-auc = 0.285573
Epoch 242
Validation prc-auc = 0.273106
Epoch 243
Validation prc-auc = 0.249464
Epoch 244
Loss = 1.3112e-01, PNorm = 43.4204, GNorm = 1.5164, lr_0 = 6.6625e-04
Validation prc-auc = 0.252675
Epoch 245
Validation prc-auc = 0.269322
Epoch 246
Validation prc-auc = 0.288808
Epoch 247
Validation prc-auc = 0.314153
Epoch 248
Validation prc-auc = 0.320777
Epoch 249
Loss = 6.6465e-02, PNorm = 43.4724, GNorm = 0.9570, lr_0 = 6.6056e-04
Validation prc-auc = 0.325519
Epoch 250
Validation prc-auc = 0.358052
Epoch 251
Validation prc-auc = 0.359315
Epoch 252
Validation prc-auc = 0.355055
Epoch 253
Validation prc-auc = 0.335141
Epoch 254
Loss = 8.0064e-02, PNorm = 43.5101, GNorm = 0.4616, lr_0 = 6.5491e-04
Validation prc-auc = 0.332598
Epoch 255
Validation prc-auc = 0.332351
Epoch 256
Validation prc-auc = 0.350230
Epoch 257
Validation prc-auc = 0.355094
Epoch 258
Validation prc-auc = 0.349684
Epoch 259
Loss = 6.4901e-02, PNorm = 43.5436, GNorm = 0.6171, lr_0 = 6.4931e-04
Validation prc-auc = 0.345407
Epoch 260
Validation prc-auc = 0.340466
Epoch 261
Validation prc-auc = 0.322335
Epoch 262
Validation prc-auc = 0.310167
Epoch 263
Validation prc-auc = 0.306942
Epoch 264
Loss = 4.2321e-02, PNorm = 43.5758, GNorm = 0.8283, lr_0 = 6.4376e-04
Validation prc-auc = 0.306834
Epoch 265
Validation prc-auc = 0.310027
Epoch 266
Validation prc-auc = 0.320669
Epoch 267
Validation prc-auc = 0.335426
Epoch 268
Validation prc-auc = 0.343004
Epoch 269
Loss = 7.4297e-02, PNorm = 43.6110, GNorm = 0.5051, lr_0 = 6.3826e-04
Validation prc-auc = 0.351927
Epoch 270
Validation prc-auc = 0.345582
Epoch 271
Validation prc-auc = 0.327389
Epoch 272
Validation prc-auc = 0.313082
Epoch 273
Validation prc-auc = 0.297424
Epoch 274
Loss = 6.4485e-02, PNorm = 43.6518, GNorm = 1.1714, lr_0 = 6.3280e-04
Validation prc-auc = 0.299880
Epoch 275
Validation prc-auc = 0.299561
Epoch 276
Validation prc-auc = 0.296439
Epoch 277
Validation prc-auc = 0.302062
Epoch 278
Validation prc-auc = 0.308998
Epoch 279
Loss = 5.7531e-02, PNorm = 43.6870, GNorm = 0.6986, lr_0 = 6.2739e-04
Validation prc-auc = 0.313560
Epoch 280
Validation prc-auc = 0.316832
Epoch 281
Validation prc-auc = 0.317869
Epoch 282
Validation prc-auc = 0.315733
Epoch 283
Validation prc-auc = 0.316716
Epoch 284
Loss = 3.7957e-02, PNorm = 43.7239, GNorm = 0.6788, lr_0 = 6.2203e-04
Validation prc-auc = 0.315142
Epoch 285
Validation prc-auc = 0.317153
Epoch 286
Validation prc-auc = 0.321649
Epoch 287
Validation prc-auc = 0.336213
Epoch 288
Validation prc-auc = 0.333429
Epoch 289
Loss = 6.0270e-02, PNorm = 43.7581, GNorm = 0.3148, lr_0 = 6.1671e-04
Validation prc-auc = 0.336819
Epoch 290
Validation prc-auc = 0.327863
Epoch 291
Validation prc-auc = 0.322373
Epoch 292
Validation prc-auc = 0.302581
Epoch 293
Validation prc-auc = 0.291757
Epoch 294
Loss = 1.0170e-01, PNorm = 43.7946, GNorm = 3.2333, lr_0 = 6.1144e-04
Validation prc-auc = 0.298997
Epoch 295
Validation prc-auc = 0.311475
Epoch 296
Validation prc-auc = 0.319769
Epoch 297
Validation prc-auc = 0.335540
Epoch 298
Validation prc-auc = 0.320175
Epoch 299
Loss = 5.9658e-02, PNorm = 43.8296, GNorm = 1.1046, lr_0 = 6.0621e-04
Validation prc-auc = 0.313462
Model 0 best validation prc-auc = 0.423829 on epoch 97
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test prc-auc = 0.439390
Ensemble test prc-auc = 0.439390
Fold 4
Splitting data with seed 4
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=1728, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=300, out_features=300, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 873,601
Moving model to cuda
Epoch 0
Validation prc-auc = 0.101002
Epoch 1
Validation prc-auc = 0.207133
Epoch 2
Validation prc-auc = 0.440870
Epoch 3
Validation prc-auc = 0.429169
Epoch 4
Loss = 6.1382e-01, PNorm = 40.7418, GNorm = 2.5564, lr_0 = 6.5000e-04
Validation prc-auc = 0.438087
Epoch 5
Validation prc-auc = 0.420445
Epoch 6
Validation prc-auc = 0.415882
Epoch 7
Validation prc-auc = 0.397114
Epoch 8
Validation prc-auc = 0.384887
Epoch 9
Loss = 5.8926e-01, PNorm = 40.7975, GNorm = 2.8700, lr_0 = 9.9743e-04
Validation prc-auc = 0.368433
Epoch 10
Validation prc-auc = 0.356165
Epoch 11
Validation prc-auc = 0.347978
Epoch 12
Validation prc-auc = 0.340144
Epoch 13
Validation prc-auc = 0.336988
Epoch 14
Loss = 5.6518e-01, PNorm = 40.8615, GNorm = 1.9114, lr_0 = 9.8890e-04
Validation prc-auc = 0.317043
Epoch 15
Validation prc-auc = 0.272321
Epoch 16
Validation prc-auc = 0.275811
Epoch 17
Validation prc-auc = 0.285641
Epoch 18
Validation prc-auc = 0.292000
Epoch 19
Loss = 5.1509e-01, PNorm = 40.9185, GNorm = 2.4351, lr_0 = 9.8045e-04
Validation prc-auc = 0.293771
Epoch 20
Validation prc-auc = 0.276469
Epoch 21
Validation prc-auc = 0.258416
Epoch 22
Validation prc-auc = 0.325707
Epoch 23
Validation prc-auc = 0.355388
Epoch 24
Loss = 5.2972e-01, PNorm = 40.9667, GNorm = 1.9558, lr_0 = 9.7207e-04
Validation prc-auc = 0.392295
Epoch 25
Validation prc-auc = 0.388539
Epoch 26
Validation prc-auc = 0.389049
Epoch 27
Validation prc-auc = 0.397229
Epoch 28
Validation prc-auc = 0.397286
Epoch 29
Loss = 4.6301e-01, PNorm = 41.0116, GNorm = 1.6805, lr_0 = 9.6376e-04
Validation prc-auc = 0.401276
Epoch 30
Validation prc-auc = 0.400051
Epoch 31
Validation prc-auc = 0.401830
Epoch 32
Validation prc-auc = 0.394634
Epoch 33
Validation prc-auc = 0.401726
Epoch 34
Loss = 4.3257e-01, PNorm = 41.0596, GNorm = 1.6812, lr_0 = 9.5552e-04
Validation prc-auc = 0.414535
Epoch 35
Validation prc-auc = 0.415848
Epoch 36
Validation prc-auc = 0.421812
Epoch 37
Validation prc-auc = 0.422871
Epoch 38
Validation prc-auc = 0.425369
Epoch 39
Loss = 4.3047e-01, PNorm = 41.1061, GNorm = 1.6830, lr_0 = 9.4735e-04
Validation prc-auc = 0.423421
Epoch 40
Validation prc-auc = 0.425221
Epoch 41
Validation prc-auc = 0.434280
Epoch 42
Validation prc-auc = 0.437169
Epoch 43
Validation prc-auc = 0.421957
Epoch 44
Loss = 3.7308e-01, PNorm = 41.1548, GNorm = 1.7477, lr_0 = 9.3925e-04
Validation prc-auc = 0.414083
Epoch 45
Validation prc-auc = 0.407336
Epoch 46
Validation prc-auc = 0.399088
Epoch 47
Validation prc-auc = 0.393427
Epoch 48
Validation prc-auc = 0.374035
Epoch 49
Loss = 3.8335e-01, PNorm = 41.2055, GNorm = 2.2316, lr_0 = 9.3122e-04
Validation prc-auc = 0.371989
Epoch 50
Validation prc-auc = 0.379827
Epoch 51
Validation prc-auc = 0.377532
Epoch 52
Validation prc-auc = 0.384419
Epoch 53
Validation prc-auc = 0.380332
Epoch 54
Loss = 3.7799e-01, PNorm = 41.2587, GNorm = 1.8962, lr_0 = 9.2326e-04
Validation prc-auc = 0.374967
Epoch 55
Validation prc-auc = 0.368024
Epoch 56
Validation prc-auc = 0.368190
Epoch 57
Validation prc-auc = 0.365325
Epoch 58
Validation prc-auc = 0.368384
Epoch 59
Loss = 3.4777e-01, PNorm = 41.3174, GNorm = 1.3521, lr_0 = 9.1537e-04
Validation prc-auc = 0.367177
Epoch 60
Validation prc-auc = 0.370070
Epoch 61
Validation prc-auc = 0.384009
Epoch 62
Validation prc-auc = 0.387354
Epoch 63
Validation prc-auc = 0.388506
Epoch 64
Loss = 3.3195e-01, PNorm = 41.3767, GNorm = 1.2673, lr_0 = 9.0754e-04
Validation prc-auc = 0.394923
Epoch 65
Validation prc-auc = 0.396253
Epoch 66
Validation prc-auc = 0.389459
Epoch 67
Validation prc-auc = 0.390311
Epoch 68
Validation prc-auc = 0.388242
Epoch 69
Loss = 3.5494e-01, PNorm = 41.4393, GNorm = 1.5588, lr_0 = 8.9978e-04
Validation prc-auc = 0.379864
Epoch 70
Validation prc-auc = 0.378730
Epoch 71
Validation prc-auc = 0.365858
Epoch 72
Validation prc-auc = 0.392704
Epoch 73
Validation prc-auc = 0.411098
Epoch 74
Loss = 2.8901e-01, PNorm = 41.5019, GNorm = 1.4618, lr_0 = 8.9209e-04
Validation prc-auc = 0.426054
Epoch 75
Validation prc-auc = 0.448927
Epoch 76
Validation prc-auc = 0.439864
Epoch 77
Validation prc-auc = 0.412736
Epoch 78
Validation prc-auc = 0.403493
Epoch 79
Loss = 4.3881e-01, PNorm = 41.5674, GNorm = 7.1318, lr_0 = 8.8447e-04
Validation prc-auc = 0.411831
Epoch 80
Validation prc-auc = 0.412386
Epoch 81
Validation prc-auc = 0.387081
Epoch 82
Validation prc-auc = 0.375586
Epoch 83
Validation prc-auc = 0.359324
Epoch 84
Loss = 2.9565e-01, PNorm = 41.6359, GNorm = 1.3232, lr_0 = 8.7691e-04
Validation prc-auc = 0.403199
Epoch 85
Validation prc-auc = 0.426493
Epoch 86
Validation prc-auc = 0.439851
Epoch 87
Validation prc-auc = 0.444122
Epoch 88
Validation prc-auc = 0.437104
Epoch 89
Loss = 2.5759e-01, PNorm = 41.7010, GNorm = 2.3250, lr_0 = 8.6941e-04
Validation prc-auc = 0.401387
Epoch 90
Validation prc-auc = 0.421283
Epoch 91
Validation prc-auc = 0.423468
Epoch 92
Validation prc-auc = 0.415910
Epoch 93
Validation prc-auc = 0.393036
Epoch 94
Loss = 2.6223e-01, PNorm = 41.7617, GNorm = 2.0350, lr_0 = 8.6198e-04
Validation prc-auc = 0.383910
Epoch 95
Validation prc-auc = 0.378566
Epoch 96
Validation prc-auc = 0.379782
Epoch 97
Validation prc-auc = 0.388460
Epoch 98
Validation prc-auc = 0.398727
Epoch 99
Loss = 2.5291e-01, PNorm = 41.8216, GNorm = 2.2769, lr_0 = 8.5461e-04
Validation prc-auc = 0.399199
Epoch 100
Validation prc-auc = 0.353385
Epoch 101
Validation prc-auc = 0.352489
Epoch 102
Validation prc-auc = 0.362946
Epoch 103
Validation prc-auc = 0.368717
Epoch 104
Loss = 2.0608e-01, PNorm = 41.8806, GNorm = 1.4149, lr_0 = 8.4730e-04
Validation prc-auc = 0.362794
Epoch 105
Validation prc-auc = 0.346693
Epoch 106
Validation prc-auc = 0.333263
Epoch 107
Validation prc-auc = 0.342855
Epoch 108
Validation prc-auc = 0.341018
Epoch 109
Loss = 1.8011e-01, PNorm = 41.9423, GNorm = 1.2711, lr_0 = 8.4006e-04
Validation prc-auc = 0.326821
Epoch 110
Validation prc-auc = 0.317990
Epoch 111
Validation prc-auc = 0.319370
Epoch 112
Validation prc-auc = 0.315265
Epoch 113
Validation prc-auc = 0.311605
Epoch 114
Loss = 2.4202e-01, PNorm = 42.0013, GNorm = 1.7310, lr_0 = 8.3288e-04
Validation prc-auc = 0.308050
Epoch 115
Validation prc-auc = 0.311673
Epoch 116
Validation prc-auc = 0.330803
Epoch 117
Validation prc-auc = 0.345445
Epoch 118
Validation prc-auc = 0.352343
Epoch 119
Loss = 2.7390e-01, PNorm = 42.0623, GNorm = 1.7485, lr_0 = 8.2576e-04
Validation prc-auc = 0.339174
Epoch 120
Validation prc-auc = 0.325655
Epoch 121
Validation prc-auc = 0.315127
Epoch 122
Validation prc-auc = 0.308499
Epoch 123
Validation prc-auc = 0.322799
Epoch 124
Loss = 1.2531e-01, PNorm = 42.1339, GNorm = 1.8182, lr_0 = 8.1870e-04
Validation prc-auc = 0.329212
Epoch 125
Validation prc-auc = 0.319304
Epoch 126
Validation prc-auc = 0.313067
Epoch 127
Validation prc-auc = 0.296525
Epoch 128
Validation prc-auc = 0.279170
Epoch 129
Loss = 1.1770e-01, PNorm = 42.1949, GNorm = 1.1153, lr_0 = 8.1170e-04
Validation prc-auc = 0.270963
Epoch 130
Validation prc-auc = 0.277682
Epoch 131
Validation prc-auc = 0.296541
Epoch 132
Validation prc-auc = 0.322653
Epoch 133
Validation prc-auc = 0.337379
Epoch 134
Loss = 2.6038e-01, PNorm = 42.2565, GNorm = 1.5375, lr_0 = 8.0476e-04
Validation prc-auc = 0.343412
Epoch 135
Validation prc-auc = 0.349817
Epoch 136
Validation prc-auc = 0.365333
Epoch 137
Validation prc-auc = 0.355908
Epoch 138
Validation prc-auc = 0.326118
Epoch 139
Loss = 1.8144e-01, PNorm = 42.3169, GNorm = 1.6509, lr_0 = 7.9788e-04
Validation prc-auc = 0.315688
Epoch 140
Validation prc-auc = 0.320499
Epoch 141
Validation prc-auc = 0.314843
Epoch 142
Validation prc-auc = 0.314775
Epoch 143
Validation prc-auc = 0.320618
Epoch 144
Loss = 1.3248e-01, PNorm = 42.3791, GNorm = 1.1224, lr_0 = 7.9106e-04
Validation prc-auc = 0.348227
Epoch 145
Validation prc-auc = 0.359283
Epoch 146
Validation prc-auc = 0.346943
Epoch 147
Validation prc-auc = 0.314227
Epoch 148
Validation prc-auc = 0.282543
Epoch 149
Loss = 1.9476e-01, PNorm = 42.4393, GNorm = 1.7677, lr_0 = 7.8430e-04
Validation prc-auc = 0.272459
Epoch 150
Validation prc-auc = 0.267117
Epoch 151
Validation prc-auc = 0.263738
Epoch 152
Validation prc-auc = 0.259727
Epoch 153
Validation prc-auc = 0.294319
Epoch 154
Loss = 8.8065e-02, PNorm = 42.5077, GNorm = 0.7727, lr_0 = 7.7759e-04
Validation prc-auc = 0.312629
Epoch 155
Validation prc-auc = 0.331539
Epoch 156
Validation prc-auc = 0.351548
Epoch 157
Validation prc-auc = 0.367745
Epoch 158
Validation prc-auc = 0.365339
Epoch 159
Loss = 1.4867e-01, PNorm = 42.5816, GNorm = 1.4418, lr_0 = 7.7095e-04
Validation prc-auc = 0.351494
Epoch 160
Validation prc-auc = 0.332446
Epoch 161
Validation prc-auc = 0.319906
Epoch 162
Validation prc-auc = 0.311463
Epoch 163
Validation prc-auc = 0.314826
Epoch 164
Loss = 1.9506e-01, PNorm = 42.6410, GNorm = 2.4079, lr_0 = 7.6436e-04
Validation prc-auc = 0.317909
Epoch 165
Validation prc-auc = 0.343789
Epoch 166
Validation prc-auc = 0.356632
Epoch 167
Validation prc-auc = 0.352078
Epoch 168
Validation prc-auc = 0.342522
Epoch 169
Loss = 5.4496e-02, PNorm = 42.6991, GNorm = 0.7204, lr_0 = 7.5782e-04
Validation prc-auc = 0.335007
Epoch 170
Validation prc-auc = 0.328921
Epoch 171
Validation prc-auc = 0.317346
Epoch 172
Validation prc-auc = 0.314768
Epoch 173
Validation prc-auc = 0.309789
Epoch 174
Loss = 1.4002e-01, PNorm = 42.7555, GNorm = 0.8947, lr_0 = 7.5134e-04
Validation prc-auc = 0.297801
Epoch 175
Validation prc-auc = 0.299601
Epoch 176
Validation prc-auc = 0.308204
Epoch 177
Validation prc-auc = 0.297701
Epoch 178
Validation prc-auc = 0.300513
Epoch 179
Loss = 9.6612e-02, PNorm = 42.7988, GNorm = 0.9060, lr_0 = 7.4492e-04
Validation prc-auc = 0.310578
Epoch 180
Validation prc-auc = 0.314319
Epoch 181
Validation prc-auc = 0.316994
Epoch 182
Validation prc-auc = 0.304166
Epoch 183
Validation prc-auc = 0.291835
Epoch 184
Loss = 5.3383e-02, PNorm = 42.8611, GNorm = 0.8250, lr_0 = 7.3855e-04
Validation prc-auc = 0.290559
Epoch 185
Validation prc-auc = 0.297139
Epoch 186
Validation prc-auc = 0.308334
Epoch 187
Validation prc-auc = 0.318825
Epoch 188
Validation prc-auc = 0.330373
Epoch 189
Loss = 1.0198e-01, PNorm = 42.9241, GNorm = 0.9576, lr_0 = 7.3224e-04
Validation prc-auc = 0.332490
Epoch 190
Validation prc-auc = 0.335888
Epoch 191
Validation prc-auc = 0.345019
Epoch 192
Validation prc-auc = 0.353070
Epoch 193
Validation prc-auc = 0.363406
Epoch 194
Loss = 1.8055e-01, PNorm = 42.9821, GNorm = 1.2653, lr_0 = 7.2598e-04
Validation prc-auc = 0.368644
Epoch 195
Validation prc-auc = 0.363111
Epoch 196
Validation prc-auc = 0.354999
Epoch 197
Validation prc-auc = 0.345695
Epoch 198
Validation prc-auc = 0.337357
Epoch 199
Loss = 8.5426e-02, PNorm = 43.0377, GNorm = 0.6138, lr_0 = 7.1977e-04
Validation prc-auc = 0.336243
Epoch 200
Validation prc-auc = 0.341011
Epoch 201
Validation prc-auc = 0.348780
Epoch 202
Validation prc-auc = 0.352912
Epoch 203
Validation prc-auc = 0.364918
Epoch 204
Loss = 1.0588e-01, PNorm = 43.0886, GNorm = 1.8714, lr_0 = 7.1362e-04
Validation prc-auc = 0.405805
Epoch 205
Validation prc-auc = 0.405899
Epoch 206
Validation prc-auc = 0.403022
Epoch 207
Validation prc-auc = 0.391186
Epoch 208
Validation prc-auc = 0.350707
Epoch 209
Loss = 8.4550e-02, PNorm = 43.1247, GNorm = 0.7893, lr_0 = 7.0752e-04
Validation prc-auc = 0.328349
Epoch 210
Validation prc-auc = 0.309168
Epoch 211
Validation prc-auc = 0.307553
Epoch 212
Validation prc-auc = 0.309243
Epoch 213
Validation prc-auc = 0.310360
Epoch 214
Loss = 7.0036e-02, PNorm = 43.1679, GNorm = 1.1797, lr_0 = 7.0147e-04
Validation prc-auc = 0.310725
Epoch 215
Validation prc-auc = 0.310068
Epoch 216
Validation prc-auc = 0.315042
Epoch 217
Validation prc-auc = 0.323419
Epoch 218
Validation prc-auc = 0.347450
Epoch 219
Loss = 8.4469e-02, PNorm = 43.2114, GNorm = 0.8854, lr_0 = 6.9548e-04
Validation prc-auc = 0.366850
Epoch 220
Validation prc-auc = 0.373738
Epoch 221
Validation prc-auc = 0.376461
Epoch 222
Validation prc-auc = 0.368701
Epoch 223
Validation prc-auc = 0.363533
Epoch 224
Loss = 7.8051e-02, PNorm = 43.2523, GNorm = 1.8431, lr_0 = 6.8953e-04
Validation prc-auc = 0.360176
Epoch 225
Validation prc-auc = 0.353844
Epoch 226
Validation prc-auc = 0.339334
Epoch 227
Validation prc-auc = 0.304477
Epoch 228
Validation prc-auc = 0.294376
Epoch 229
Loss = 9.1442e-02, PNorm = 43.2919, GNorm = 1.0744, lr_0 = 6.8364e-04
Validation prc-auc = 0.292159
Epoch 230
Validation prc-auc = 0.293500
Epoch 231
Validation prc-auc = 0.304071
Epoch 232
Validation prc-auc = 0.312512
Epoch 233
Validation prc-auc = 0.320174
Epoch 234
Loss = 6.5324e-02, PNorm = 43.3403, GNorm = 1.6076, lr_0 = 6.7779e-04
Validation prc-auc = 0.326794
Epoch 235
Validation prc-auc = 0.317848
Epoch 236
Validation prc-auc = 0.316792
Epoch 237
Validation prc-auc = 0.314233
Epoch 238
Validation prc-auc = 0.317941
Epoch 239
Loss = 5.8915e-02, PNorm = 43.3846, GNorm = 1.5780, lr_0 = 6.7200e-04
Validation prc-auc = 0.317656
Epoch 240
Validation prc-auc = 0.311421
Epoch 241
Validation prc-auc = 0.317155
Epoch 242
Validation prc-auc = 0.331517
Epoch 243
Validation prc-auc = 0.337942
Epoch 244
Loss = 1.1249e-01, PNorm = 43.4437, GNorm = 1.6398, lr_0 = 6.6625e-04
Validation prc-auc = 0.366885
Epoch 245
Validation prc-auc = 0.384223
Epoch 246
Validation prc-auc = 0.378214
Epoch 247
Validation prc-auc = 0.363352
Epoch 248
Validation prc-auc = 0.353146
Epoch 249
Loss = 6.9361e-02, PNorm = 43.4941, GNorm = 0.6519, lr_0 = 6.6056e-04
Validation prc-auc = 0.341018
Epoch 250
Validation prc-auc = 0.338051
Epoch 251
Validation prc-auc = 0.352814
Epoch 252
Validation prc-auc = 0.374792
Epoch 253
Validation prc-auc = 0.381353
Epoch 254
Loss = 1.4029e-01, PNorm = 43.5392, GNorm = 1.6461, lr_0 = 6.5491e-04
Validation prc-auc = 0.366185
Epoch 255
Validation prc-auc = 0.341957
Epoch 256
Validation prc-auc = 0.333055
Epoch 257
Validation prc-auc = 0.325720
Epoch 258
Validation prc-auc = 0.327650
Epoch 259
Loss = 2.3077e-01, PNorm = 43.5818, GNorm = 0.4188, lr_0 = 6.4931e-04
Validation prc-auc = 0.328978
Epoch 260
Validation prc-auc = 0.333497
Epoch 261
Validation prc-auc = 0.331955
Epoch 262
Validation prc-auc = 0.328293
Epoch 263
Validation prc-auc = 0.333912
Epoch 264
Loss = 7.5365e-02, PNorm = 43.6286, GNorm = 1.5654, lr_0 = 6.4376e-04
Validation prc-auc = 0.342561
Epoch 265
Validation prc-auc = 0.341882
Epoch 266
Validation prc-auc = 0.338705
Epoch 267
Validation prc-auc = 0.335867
Epoch 268
Validation prc-auc = 0.345183
Epoch 269
Loss = 2.0902e-01, PNorm = 43.6673, GNorm = 2.2765, lr_0 = 6.3826e-04
Validation prc-auc = 0.353213
Epoch 270
Validation prc-auc = 0.350482
Epoch 271
Validation prc-auc = 0.340023
Epoch 272
Validation prc-auc = 0.339265
Epoch 273
Validation prc-auc = 0.344906
Epoch 274
Loss = 3.7632e-02, PNorm = 43.7244, GNorm = 0.6993, lr_0 = 6.3280e-04
Validation prc-auc = 0.350566
Epoch 275
Validation prc-auc = 0.353812
Epoch 276
Validation prc-auc = 0.372551
Epoch 277
Validation prc-auc = 0.371589
Epoch 278
Validation prc-auc = 0.371183
Epoch 279
Loss = 1.3375e-01, PNorm = 43.7766, GNorm = 1.0356, lr_0 = 6.2739e-04
Validation prc-auc = 0.371562
Epoch 280
Validation prc-auc = 0.348237
Epoch 281
Validation prc-auc = 0.330950
Epoch 282
Validation prc-auc = 0.330038
Epoch 283
Validation prc-auc = 0.345704
Epoch 284
Loss = 4.2410e-02, PNorm = 43.8146, GNorm = 0.9750, lr_0 = 6.2203e-04
Validation prc-auc = 0.355135
Epoch 285
Validation prc-auc = 0.351422
Epoch 286
Validation prc-auc = 0.346547
Epoch 287
Validation prc-auc = 0.351906
Epoch 288
Validation prc-auc = 0.378896
Epoch 289
Loss = 6.5442e-02, PNorm = 43.8502, GNorm = 1.5225, lr_0 = 6.1671e-04
Validation prc-auc = 0.378922
Epoch 290
Validation prc-auc = 0.353451
Epoch 291
Validation prc-auc = 0.352533
Epoch 292
Validation prc-auc = 0.354006
Epoch 293
Validation prc-auc = 0.352662
Epoch 294
Loss = 9.0046e-02, PNorm = 43.8846, GNorm = 1.5731, lr_0 = 6.1144e-04
Validation prc-auc = 0.373039
Epoch 295
Validation prc-auc = 0.374380
Epoch 296
Validation prc-auc = 0.381759
Epoch 297
Validation prc-auc = 0.334999
Epoch 298
Validation prc-auc = 0.318824
Epoch 299
Loss = 5.0261e-02, PNorm = 43.9176, GNorm = 1.1431, lr_0 = 6.0621e-04
Validation prc-auc = 0.313918
Model 0 best validation prc-auc = 0.448927 on epoch 75
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test prc-auc = 0.404111
Ensemble test prc-auc = 0.404111
Fold 5
Splitting data with seed 5
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=1728, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=300, out_features=300, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 873,601
Moving model to cuda
Epoch 0
Validation prc-auc = 0.103010
Epoch 1
Validation prc-auc = 0.247788
Epoch 2
Validation prc-auc = 0.429709
Epoch 3
Validation prc-auc = 0.444380
Epoch 4
Loss = 6.6035e-01, PNorm = 40.7428, GNorm = 3.4719, lr_0 = 6.5000e-04
Validation prc-auc = 0.449125
Epoch 5
Validation prc-auc = 0.459808
Epoch 6
Validation prc-auc = 0.442710
Epoch 7
Validation prc-auc = 0.456946
Epoch 8
Validation prc-auc = 0.431913
Epoch 9
Loss = 5.6177e-01, PNorm = 40.7980, GNorm = 2.4786, lr_0 = 9.9743e-04
Validation prc-auc = 0.421496
Epoch 10
Validation prc-auc = 0.412718
Epoch 11
Validation prc-auc = 0.407483
Epoch 12
Validation prc-auc = 0.353719
Epoch 13
Validation prc-auc = 0.258515
Epoch 14
Loss = 5.8774e-01, PNorm = 40.8608, GNorm = 2.4098, lr_0 = 9.8890e-04
Validation prc-auc = 0.258673
Epoch 15
Validation prc-auc = 0.266158
Epoch 16
Validation prc-auc = 0.379894
Epoch 17
Validation prc-auc = 0.395369
Epoch 18
Validation prc-auc = 0.405626
Epoch 19
Loss = 4.9702e-01, PNorm = 40.9161, GNorm = 3.8377, lr_0 = 9.8045e-04
Validation prc-auc = 0.407938
Epoch 20
Validation prc-auc = 0.405611
Epoch 21
Validation prc-auc = 0.409831
Epoch 22
Validation prc-auc = 0.410326
Epoch 23
Validation prc-auc = 0.409989
Epoch 24
Loss = 5.5690e-01, PNorm = 40.9652, GNorm = 1.9144, lr_0 = 9.7207e-04
Validation prc-auc = 0.412122
Epoch 25
Validation prc-auc = 0.370878
Epoch 26
Validation prc-auc = 0.360857
Epoch 27
Validation prc-auc = 0.351664
Epoch 28
Validation prc-auc = 0.348101
Epoch 29
Loss = 4.9987e-01, PNorm = 41.0095, GNorm = 2.1765, lr_0 = 9.6376e-04
Validation prc-auc = 0.344897
Epoch 30
Validation prc-auc = 0.337345
Epoch 31
Validation prc-auc = 0.331500
Epoch 32
Validation prc-auc = 0.313595
Epoch 33
Validation prc-auc = 0.333558
Epoch 34
Loss = 4.1878e-01, PNorm = 41.0535, GNorm = 1.5638, lr_0 = 9.5552e-04
Validation prc-auc = 0.360711
Epoch 35
Validation prc-auc = 0.371873
Epoch 36
Validation prc-auc = 0.365798
Epoch 37
Validation prc-auc = 0.357314
Epoch 38
Validation prc-auc = 0.354886
Epoch 39
Loss = 4.6617e-01, PNorm = 41.1011, GNorm = 1.7734, lr_0 = 9.4735e-04
Validation prc-auc = 0.357648
Epoch 40
Validation prc-auc = 0.362060
Epoch 41
Validation prc-auc = 0.365095
Epoch 42
Validation prc-auc = 0.364971
Epoch 43
Validation prc-auc = 0.371712
Epoch 44
Loss = 4.6161e-01, PNorm = 41.1527, GNorm = 2.0097, lr_0 = 9.3925e-04
Validation prc-auc = 0.372542
Epoch 45
Validation prc-auc = 0.361286
Epoch 46
Validation prc-auc = 0.357313
Epoch 47
Validation prc-auc = 0.361034
Epoch 48
Validation prc-auc = 0.367836
Epoch 49
Loss = 3.7415e-01, PNorm = 41.2049, GNorm = 1.9903, lr_0 = 9.3122e-04
Validation prc-auc = 0.375251
Epoch 50
Validation prc-auc = 0.383474
Epoch 51
Validation prc-auc = 0.374168
Epoch 52
Validation prc-auc = 0.387383
Epoch 53
Validation prc-auc = 0.385507
Epoch 54
Loss = 3.6880e-01, PNorm = 41.2599, GNorm = 2.1380, lr_0 = 9.2326e-04
Validation prc-auc = 0.382918
Epoch 55
Validation prc-auc = 0.383491
Epoch 56
Validation prc-auc = 0.379661
Epoch 57
Validation prc-auc = 0.379636
Epoch 58
Validation prc-auc = 0.381524
Epoch 59
Loss = 2.9148e-01, PNorm = 41.3154, GNorm = 1.3943, lr_0 = 9.1537e-04
Validation prc-auc = 0.377604
Epoch 60
Validation prc-auc = 0.371746
Epoch 61
Validation prc-auc = 0.383892
Epoch 62
Validation prc-auc = 0.370605
Epoch 63
Validation prc-auc = 0.361681
Epoch 64
Loss = 3.2634e-01, PNorm = 41.3736, GNorm = 1.3949, lr_0 = 9.0754e-04
Validation prc-auc = 0.357460
Epoch 65
Validation prc-auc = 0.353192
Epoch 66
Validation prc-auc = 0.356574
Epoch 67
Validation prc-auc = 0.359754
Epoch 68
Validation prc-auc = 0.364134
Epoch 69
Loss = 3.1931e-01, PNorm = 41.4334, GNorm = 1.8179, lr_0 = 8.9978e-04
Validation prc-auc = 0.372049
Epoch 70
Validation prc-auc = 0.377407
Epoch 71
Validation prc-auc = 0.376612
Epoch 72
Validation prc-auc = 0.373114
Epoch 73
Validation prc-auc = 0.363189
Epoch 74
Loss = 2.7905e-01, PNorm = 41.4916, GNorm = 1.7386, lr_0 = 8.9209e-04
Validation prc-auc = 0.358043
Epoch 75
Validation prc-auc = 0.359746
Epoch 76
Validation prc-auc = 0.312495
Epoch 77
Validation prc-auc = 0.337879
Epoch 78
Validation prc-auc = 0.353626
Epoch 79
Loss = 2.6352e-01, PNorm = 41.5588, GNorm = 1.6378, lr_0 = 8.8447e-04
Validation prc-auc = 0.369615
Epoch 80
Validation prc-auc = 0.387148
Epoch 81
Validation prc-auc = 0.407331
Epoch 82
Validation prc-auc = 0.424208
Epoch 83
Validation prc-auc = 0.433142
Epoch 84
Loss = 2.3304e-01, PNorm = 41.6236, GNorm = 2.0111, lr_0 = 8.7691e-04
Validation prc-auc = 0.427255
Epoch 85
Validation prc-auc = 0.427292
Epoch 86
Validation prc-auc = 0.426847
Epoch 87
Validation prc-auc = 0.429424
Epoch 88
Validation prc-auc = 0.414103
Epoch 89
Loss = 3.5434e-01, PNorm = 41.6926, GNorm = 2.5187, lr_0 = 8.6941e-04
Validation prc-auc = 0.387036
Epoch 90
Validation prc-auc = 0.351403
Epoch 91
Validation prc-auc = 0.355180
Epoch 92
Validation prc-auc = 0.382723
Epoch 93
Validation prc-auc = 0.389211
Epoch 94
Loss = 2.6317e-01, PNorm = 41.7640, GNorm = 1.9861, lr_0 = 8.6198e-04
Validation prc-auc = 0.400826
Epoch 95
Validation prc-auc = 0.400398
Epoch 96
Validation prc-auc = 0.398371
Epoch 97
Validation prc-auc = 0.395477
Epoch 98
Validation prc-auc = 0.374893
Epoch 99
Loss = 2.0847e-01, PNorm = 41.8322, GNorm = 1.3594, lr_0 = 8.5461e-04
Validation prc-auc = 0.363315
Epoch 100
Validation prc-auc = 0.392323
Epoch 101
Validation prc-auc = 0.392014
Epoch 102
Validation prc-auc = 0.379256
Epoch 103
Validation prc-auc = 0.377443
Epoch 104
Loss = 2.2167e-01, PNorm = 41.8974, GNorm = 1.7091, lr_0 = 8.4730e-04
Validation prc-auc = 0.355294
Epoch 105
Validation prc-auc = 0.347483
Epoch 106
Validation prc-auc = 0.343259
Epoch 107
Validation prc-auc = 0.356162
Epoch 108
Validation prc-auc = 0.354068
Epoch 109
Loss = 2.2743e-01, PNorm = 41.9596, GNorm = 2.4640, lr_0 = 8.4006e-04
Validation prc-auc = 0.337398
Epoch 110
Validation prc-auc = 0.309771
Epoch 111
Validation prc-auc = 0.308519
Epoch 112
Validation prc-auc = 0.337912
Epoch 113
Validation prc-auc = 0.359773
Epoch 114
Loss = 1.6893e-01, PNorm = 42.0187, GNorm = 1.7577, lr_0 = 8.3288e-04
Validation prc-auc = 0.362750
Epoch 115
Validation prc-auc = 0.364478
Epoch 116
Validation prc-auc = 0.375715
Epoch 117
Validation prc-auc = 0.389474
Epoch 118
Validation prc-auc = 0.386860
Epoch 119
Loss = 1.1696e-01, PNorm = 42.0793, GNorm = 1.2026, lr_0 = 8.2576e-04
Validation prc-auc = 0.366314
Epoch 120
Validation prc-auc = 0.342260
Epoch 121
Validation prc-auc = 0.299578
Epoch 122
Validation prc-auc = 0.294614
Epoch 123
Validation prc-auc = 0.308319
Epoch 124
Loss = 1.4464e-01, PNorm = 42.1362, GNorm = 1.5392, lr_0 = 8.1870e-04
Validation prc-auc = 0.324025
Epoch 125
Validation prc-auc = 0.348420
Epoch 126
Validation prc-auc = 0.353970
Epoch 127
Validation prc-auc = 0.343841
Epoch 128
Validation prc-auc = 0.364237
Epoch 129
Loss = 2.0269e-01, PNorm = 42.2000, GNorm = 1.4403, lr_0 = 8.1170e-04
Validation prc-auc = 0.367072
Epoch 130
Validation prc-auc = 0.336184
Epoch 131
Validation prc-auc = 0.327052
Epoch 132
Validation prc-auc = 0.318619
Epoch 133
Validation prc-auc = 0.325576
Epoch 134
Loss = 1.5775e-01, PNorm = 42.2671, GNorm = 1.1857, lr_0 = 8.0476e-04
Validation prc-auc = 0.349933
Epoch 135
Validation prc-auc = 0.348979
Epoch 136
Validation prc-auc = 0.354908
Epoch 137
Validation prc-auc = 0.383498
Epoch 138
Validation prc-auc = 0.368964
Epoch 139
Loss = 7.7435e-02, PNorm = 42.3289, GNorm = 0.6837, lr_0 = 7.9788e-04
Validation prc-auc = 0.337489
Epoch 140
Validation prc-auc = 0.319927
Epoch 141
Validation prc-auc = 0.306849
Epoch 142
Validation prc-auc = 0.307281
Epoch 143
Validation prc-auc = 0.307457
Epoch 144
Loss = 1.6553e-01, PNorm = 42.3858, GNorm = 0.7101, lr_0 = 7.9106e-04
Validation prc-auc = 0.320201
Epoch 145
Validation prc-auc = 0.347743
Epoch 146
Validation prc-auc = 0.356682
Epoch 147
Validation prc-auc = 0.336893
Epoch 148
Validation prc-auc = 0.328635
Epoch 149
Loss = 2.3385e-01, PNorm = 42.4384, GNorm = 2.1987, lr_0 = 7.8430e-04
Validation prc-auc = 0.302809
Epoch 150
Validation prc-auc = 0.308315
Epoch 151
Validation prc-auc = 0.300048
Epoch 152
Validation prc-auc = 0.281399
Epoch 153
Validation prc-auc = 0.283889
Epoch 154
Loss = 1.0926e-01, PNorm = 42.4967, GNorm = 1.6498, lr_0 = 7.7759e-04
Validation prc-auc = 0.286523
Epoch 155
Validation prc-auc = 0.289151
Epoch 156
Validation prc-auc = 0.295127
Epoch 157
Validation prc-auc = 0.326709
Epoch 158
Validation prc-auc = 0.321948
Epoch 159
Loss = 7.6843e-02, PNorm = 42.5580, GNorm = 0.8184, lr_0 = 7.7095e-04
Validation prc-auc = 0.329339
Epoch 160
Validation prc-auc = 0.330363
Epoch 161
Validation prc-auc = 0.321597
Epoch 162
Validation prc-auc = 0.340688
Epoch 163
Validation prc-auc = 0.315899
Epoch 164
Loss = 1.6312e-01, PNorm = 42.6160, GNorm = 1.6583, lr_0 = 7.6436e-04
Validation prc-auc = 0.316082
Epoch 165
Validation prc-auc = 0.312893
Epoch 166
Validation prc-auc = 0.313754
Epoch 167
Validation prc-auc = 0.316670
Epoch 168
Validation prc-auc = 0.320813
Epoch 169
Loss = 1.0536e-01, PNorm = 42.6679, GNorm = 0.9362, lr_0 = 7.5782e-04
Validation prc-auc = 0.320532
Epoch 170
Validation prc-auc = 0.321355
Epoch 171
Validation prc-auc = 0.317088
Epoch 172
Validation prc-auc = 0.314971
Epoch 173
Validation prc-auc = 0.316428
Epoch 174
Loss = 1.1084e-01, PNorm = 42.7291, GNorm = 1.1018, lr_0 = 7.5134e-04
Validation prc-auc = 0.282352
Epoch 175
Validation prc-auc = 0.274435
Epoch 176
Validation prc-auc = 0.279260
Epoch 177
Validation prc-auc = 0.286316
Epoch 178
Validation prc-auc = 0.356960
Epoch 179
Loss = 1.0669e-01, PNorm = 42.7834, GNorm = 1.2485, lr_0 = 7.4492e-04
Validation prc-auc = 0.385830
Epoch 180
Validation prc-auc = 0.360996
Epoch 181
Validation prc-auc = 0.345629
Epoch 182
Validation prc-auc = 0.332061
Epoch 183
Validation prc-auc = 0.340517
Epoch 184
Loss = 1.6860e-01, PNorm = 42.8488, GNorm = 2.0276, lr_0 = 7.3855e-04
Validation prc-auc = 0.329308
Epoch 185
Validation prc-auc = 0.329345
Epoch 186
Validation prc-auc = 0.343975
Epoch 187
Validation prc-auc = 0.336356
Epoch 188
Validation prc-auc = 0.337229
Epoch 189
Loss = 1.2183e-01, PNorm = 42.9221, GNorm = 0.7339, lr_0 = 7.3224e-04
Validation prc-auc = 0.342876
Epoch 190
Validation prc-auc = 0.371070
Epoch 191
Validation prc-auc = 0.380296
Epoch 192
Validation prc-auc = 0.379196
Epoch 193
Validation prc-auc = 0.370431
Epoch 194
Loss = 1.6783e-01, PNorm = 42.9790, GNorm = 1.2150, lr_0 = 7.2598e-04
Validation prc-auc = 0.360381
Epoch 195
Validation prc-auc = 0.366144
Epoch 196
Validation prc-auc = 0.374244
Epoch 197
Validation prc-auc = 0.376086
Epoch 198
Validation prc-auc = 0.357822
Epoch 199
Loss = 1.2876e-01, PNorm = 43.0411, GNorm = 1.8714, lr_0 = 7.1977e-04
Validation prc-auc = 0.334357
Epoch 200
Validation prc-auc = 0.323799
Epoch 201
Validation prc-auc = 0.329348
Epoch 202
Validation prc-auc = 0.334114
Epoch 203
Validation prc-auc = 0.338503
Epoch 204
Loss = 9.4608e-02, PNorm = 43.1094, GNorm = 1.3040, lr_0 = 7.1362e-04
Validation prc-auc = 0.367106
Epoch 205
Validation prc-auc = 0.389403
Epoch 206
Validation prc-auc = 0.386720
Epoch 207
Validation prc-auc = 0.385875
Epoch 208
Validation prc-auc = 0.384277
Epoch 209
Loss = 5.7272e-02, PNorm = 43.1648, GNorm = 0.4891, lr_0 = 7.0752e-04
Validation prc-auc = 0.365980
Epoch 210
Validation prc-auc = 0.368185
Epoch 211
Validation prc-auc = 0.387922
Epoch 212
Validation prc-auc = 0.390832
Epoch 213
Validation prc-auc = 0.392406
Epoch 214
Loss = 7.3894e-02, PNorm = 43.2178, GNorm = 1.0045, lr_0 = 7.0147e-04
Validation prc-auc = 0.369419
Epoch 215
Validation prc-auc = 0.364070
Epoch 216
Validation prc-auc = 0.326069
Epoch 217
Validation prc-auc = 0.319274
Epoch 218
Validation prc-auc = 0.292610
Epoch 219
Loss = 1.0803e-01, PNorm = 43.2738, GNorm = 1.4410, lr_0 = 6.9548e-04
Validation prc-auc = 0.289880
Epoch 220
Validation prc-auc = 0.324157
Epoch 221
Validation prc-auc = 0.342364
Epoch 222
Validation prc-auc = 0.360137
Epoch 223
Validation prc-auc = 0.380420
Epoch 224
Loss = 3.5227e-02, PNorm = 43.3223, GNorm = 0.5921, lr_0 = 6.8953e-04
Validation prc-auc = 0.382378
Epoch 225
Validation prc-auc = 0.356749
Epoch 226
Validation prc-auc = 0.352977
Epoch 227
Validation prc-auc = 0.350723
Epoch 228
Validation prc-auc = 0.349066
Epoch 229
Loss = 1.2716e-01, PNorm = 43.3639, GNorm = 0.7631, lr_0 = 6.8364e-04
Validation prc-auc = 0.350463
Epoch 230
Validation prc-auc = 0.350958
Epoch 231
Validation prc-auc = 0.333092
Epoch 232
Validation prc-auc = 0.319089
Epoch 233
Validation prc-auc = 0.309388
Epoch 234
Loss = 9.1355e-02, PNorm = 43.4027, GNorm = 0.6925, lr_0 = 6.7779e-04
Validation prc-auc = 0.307789
Epoch 235
Validation prc-auc = 0.277876
Epoch 236
Validation prc-auc = 0.325748
Epoch 237
Validation prc-auc = 0.351698
Epoch 238
Validation prc-auc = 0.349557
Epoch 239
Loss = 8.1077e-02, PNorm = 43.4512, GNorm = 1.2753, lr_0 = 6.7200e-04
Validation prc-auc = 0.355471
Epoch 240
Validation prc-auc = 0.362331
Epoch 241
Validation prc-auc = 0.346024
Epoch 242
Validation prc-auc = 0.346397
Epoch 243
Validation prc-auc = 0.345309
Epoch 244
Loss = 1.3820e-01, PNorm = 43.4937, GNorm = 1.5394, lr_0 = 6.6625e-04
Validation prc-auc = 0.324822
Epoch 245
Validation prc-auc = 0.306054
Epoch 246
Validation prc-auc = 0.303507
Epoch 247
Validation prc-auc = 0.302464
Epoch 248
Validation prc-auc = 0.307487
Epoch 249
Loss = 8.7215e-02, PNorm = 43.5365, GNorm = 1.7454, lr_0 = 6.6056e-04
Validation prc-auc = 0.311589
Epoch 250
Validation prc-auc = 0.317728
Epoch 251
Validation prc-auc = 0.322296
Epoch 252
Validation prc-auc = 0.331216
Epoch 253
Validation prc-auc = 0.344901
Epoch 254
Loss = 1.0164e-01, PNorm = 43.5790, GNorm = 1.7881, lr_0 = 6.5491e-04
Validation prc-auc = 0.365329
Epoch 255
Validation prc-auc = 0.365449
Epoch 256
Validation prc-auc = 0.335972
Epoch 257
Validation prc-auc = 0.330013
Epoch 258
Validation prc-auc = 0.336123
Epoch 259
Loss = 6.0886e-02, PNorm = 43.6256, GNorm = 0.6667, lr_0 = 6.4931e-04
Validation prc-auc = 0.327132
Epoch 260
Validation prc-auc = 0.317645
Epoch 261
Validation prc-auc = 0.311147
Epoch 262
Validation prc-auc = 0.313147
Epoch 263
Validation prc-auc = 0.308195
Epoch 264
Loss = 1.2480e-01, PNorm = 43.6656, GNorm = 1.8950, lr_0 = 6.4376e-04
Validation prc-auc = 0.314465
Epoch 265
Validation prc-auc = 0.327505
Epoch 266
Validation prc-auc = 0.381960
Epoch 267
Validation prc-auc = 0.386699
Epoch 268
Validation prc-auc = 0.361900
Epoch 269
Loss = 1.6674e-01, PNorm = 43.7071, GNorm = 1.7442, lr_0 = 6.3826e-04
Validation prc-auc = 0.349246
Epoch 270
Validation prc-auc = 0.354960
Epoch 271
Validation prc-auc = 0.377824
Epoch 272
Validation prc-auc = 0.393156
Epoch 273
Validation prc-auc = 0.378942
Epoch 274
Loss = 3.8591e-02, PNorm = 43.7453, GNorm = 0.8861, lr_0 = 6.3280e-04
Validation prc-auc = 0.352825
Epoch 275
Validation prc-auc = 0.356896
Epoch 276
Validation prc-auc = 0.394543
Epoch 277
Validation prc-auc = 0.412886
Epoch 278
Validation prc-auc = 0.398766
Epoch 279
Loss = 6.3613e-02, PNorm = 43.7777, GNorm = 1.0315, lr_0 = 6.2739e-04
Validation prc-auc = 0.399252
Epoch 280
Validation prc-auc = 0.348603
Epoch 281
Validation prc-auc = 0.326699
Epoch 282
Validation prc-auc = 0.317253
Epoch 283
Validation prc-auc = 0.315788
Epoch 284
Loss = 2.0287e-01, PNorm = 43.8122, GNorm = 1.7229, lr_0 = 6.2203e-04
Validation prc-auc = 0.323334
Epoch 285
Validation prc-auc = 0.333322
Epoch 286
Validation prc-auc = 0.369427
Epoch 287
Validation prc-auc = 0.385808
Epoch 288
Validation prc-auc = 0.374203
Epoch 289
Loss = 3.4162e-02, PNorm = 43.8522, GNorm = 0.4341, lr_0 = 6.1671e-04
Validation prc-auc = 0.345402
Epoch 290
Validation prc-auc = 0.358702
Epoch 291
Validation prc-auc = 0.358343
Epoch 292
Validation prc-auc = 0.354585
Epoch 293
Validation prc-auc = 0.358916
Epoch 294
Loss = 9.8019e-02, PNorm = 43.8885, GNorm = 1.7772, lr_0 = 6.1144e-04
Validation prc-auc = 0.335149
Epoch 295
Validation prc-auc = 0.328497
Epoch 296
Validation prc-auc = 0.318504
Epoch 297
Validation prc-auc = 0.278354
Epoch 298
Validation prc-auc = 0.279222
Epoch 299
Loss = 6.2964e-02, PNorm = 43.9252, GNorm = 1.3004, lr_0 = 6.0621e-04
Validation prc-auc = 0.274618
Model 0 best validation prc-auc = 0.459808 on epoch 5
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test prc-auc = 0.422134
Ensemble test prc-auc = 0.422134
Fold 6
Splitting data with seed 6
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=1728, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=300, out_features=300, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 873,601
Moving model to cuda
Epoch 0
Validation prc-auc = 0.096190
Epoch 1
Validation prc-auc = 0.149020
Epoch 2
Validation prc-auc = 0.378959
Epoch 3
Validation prc-auc = 0.427484
Epoch 4
Loss = 6.3739e-01, PNorm = 40.7418, GNorm = 2.6862, lr_0 = 6.5000e-04
Validation prc-auc = 0.433383
Epoch 5
Validation prc-auc = 0.403212
Epoch 6
Validation prc-auc = 0.397752
Epoch 7
Validation prc-auc = 0.387818
Epoch 8
Validation prc-auc = 0.387525
Epoch 9
Loss = 5.5127e-01, PNorm = 40.7979, GNorm = 2.3775, lr_0 = 9.9743e-04
Validation prc-auc = 0.396521
Epoch 10
Validation prc-auc = 0.385600
Epoch 11
Validation prc-auc = 0.395690
Epoch 12
Validation prc-auc = 0.365362
Epoch 13
Validation prc-auc = 0.364770
Epoch 14
Loss = 7.1545e-01, PNorm = 40.8602, GNorm = 6.1286, lr_0 = 9.8890e-04
Validation prc-auc = 0.360015
Epoch 15
Validation prc-auc = 0.353686
Epoch 16
Validation prc-auc = 0.341497
Epoch 17
Validation prc-auc = 0.338801
Epoch 18
Validation prc-auc = 0.339764
Epoch 19
Loss = 5.7496e-01, PNorm = 40.9115, GNorm = 2.6985, lr_0 = 9.8045e-04
Validation prc-auc = 0.340440
Epoch 20
Validation prc-auc = 0.342944
Epoch 21
Validation prc-auc = 0.353121
Epoch 22
Validation prc-auc = 0.372158
Epoch 23
Validation prc-auc = 0.384996
Epoch 24
Loss = 4.9236e-01, PNorm = 40.9567, GNorm = 1.7169, lr_0 = 9.7207e-04
Validation prc-auc = 0.399030
Epoch 25
Validation prc-auc = 0.397870
Epoch 26
Validation prc-auc = 0.393356
Epoch 27
Validation prc-auc = 0.389766
Epoch 28
Validation prc-auc = 0.387428
Epoch 29
Loss = 4.3737e-01, PNorm = 40.9978, GNorm = 1.4557, lr_0 = 9.6376e-04
Validation prc-auc = 0.383639
Epoch 30
Validation prc-auc = 0.369257
Epoch 31
Validation prc-auc = 0.362181
Epoch 32
Validation prc-auc = 0.373714
Epoch 33
Validation prc-auc = 0.358186
Epoch 34
Loss = 4.7606e-01, PNorm = 41.0382, GNorm = 1.6402, lr_0 = 9.5552e-04
Validation prc-auc = 0.349463
Epoch 35
Validation prc-auc = 0.340677
Epoch 36
Validation prc-auc = 0.332882
Epoch 37
Validation prc-auc = 0.336142
Epoch 38
Validation prc-auc = 0.347310
Epoch 39
Loss = 4.8583e-01, PNorm = 41.0820, GNorm = 2.0475, lr_0 = 9.4735e-04
Validation prc-auc = 0.367739
Epoch 40
Validation prc-auc = 0.379770
Epoch 41
Validation prc-auc = 0.389858
Epoch 42
Validation prc-auc = 0.381560
Epoch 43
Validation prc-auc = 0.381947
Epoch 44
Loss = 4.0661e-01, PNorm = 41.1301, GNorm = 1.9589, lr_0 = 9.3925e-04
Validation prc-auc = 0.376554
Epoch 45
Validation prc-auc = 0.377389
Epoch 46
Validation prc-auc = 0.380442
Epoch 47
Validation prc-auc = 0.384787
Epoch 48
Validation prc-auc = 0.385901
Epoch 49
Loss = 3.7915e-01, PNorm = 41.1810, GNorm = 1.4388, lr_0 = 9.3122e-04
Validation prc-auc = 0.386414
Epoch 50
Validation prc-auc = 0.385964
Epoch 51
Validation prc-auc = 0.375629
Epoch 52
Validation prc-auc = 0.365140
Epoch 53
Validation prc-auc = 0.363084
Epoch 54
Loss = 3.4769e-01, PNorm = 41.2321, GNorm = 1.5928, lr_0 = 9.2326e-04
Validation prc-auc = 0.338673
Epoch 55
Validation prc-auc = 0.308008
Epoch 56
Validation prc-auc = 0.305127
Epoch 57
Validation prc-auc = 0.305336
Epoch 58
Validation prc-auc = 0.312118
Epoch 59
Loss = 3.2390e-01, PNorm = 41.2886, GNorm = 1.3608, lr_0 = 9.1537e-04
Validation prc-auc = 0.325608
Epoch 60
Validation prc-auc = 0.335844
Epoch 61
Validation prc-auc = 0.356931
Epoch 62
Validation prc-auc = 0.355352
Epoch 63
Validation prc-auc = 0.354200
Epoch 64
Loss = 3.8289e-01, PNorm = 41.3535, GNorm = 2.1975, lr_0 = 9.0754e-04
Validation prc-auc = 0.350897
Epoch 65
Validation prc-auc = 0.350339
Epoch 66
Validation prc-auc = 0.348325
Epoch 67
Validation prc-auc = 0.369853
Epoch 68
Validation prc-auc = 0.379145
Epoch 69
Loss = 3.0935e-01, PNorm = 41.4173, GNorm = 2.0623, lr_0 = 8.9978e-04
Validation prc-auc = 0.384424
Epoch 70
Validation prc-auc = 0.387673
Epoch 71
Validation prc-auc = 0.389297
Epoch 72
Validation prc-auc = 0.393461
Epoch 73
Validation prc-auc = 0.386494
Epoch 74
Loss = 2.7118e-01, PNorm = 41.4778, GNorm = 1.6087, lr_0 = 8.9209e-04
Validation prc-auc = 0.381788
Epoch 75
Validation prc-auc = 0.380003
Epoch 76
Validation prc-auc = 0.377028
Epoch 77
Validation prc-auc = 0.381153
Epoch 78
Validation prc-auc = 0.396409
Epoch 79
Loss = 2.5512e-01, PNorm = 41.5447, GNorm = 1.0905, lr_0 = 8.8447e-04
Validation prc-auc = 0.412766
Epoch 80
Validation prc-auc = 0.421041
Epoch 81
Validation prc-auc = 0.417938
Epoch 82
Validation prc-auc = 0.398753
Epoch 83
Validation prc-auc = 0.344288
Epoch 84
Loss = 2.6094e-01, PNorm = 41.6060, GNorm = 1.9340, lr_0 = 8.7691e-04
Validation prc-auc = 0.342619
Epoch 85
Validation prc-auc = 0.361021
Epoch 86
Validation prc-auc = 0.363342
Epoch 87
Validation prc-auc = 0.375222
Epoch 88
Validation prc-auc = 0.374824
Epoch 89
Loss = 2.5962e-01, PNorm = 41.6701, GNorm = 1.4494, lr_0 = 8.6941e-04
Validation prc-auc = 0.379548
Epoch 90
Validation prc-auc = 0.366261
Epoch 91
Validation prc-auc = 0.378329
Epoch 92
Validation prc-auc = 0.375557
Epoch 93
Validation prc-auc = 0.379411
Epoch 94
Loss = 2.6617e-01, PNorm = 41.7375, GNorm = 1.2739, lr_0 = 8.6198e-04
Validation prc-auc = 0.385800
Epoch 95
Validation prc-auc = 0.382922
Epoch 96
Validation prc-auc = 0.344083
Epoch 97
Validation prc-auc = 0.342610
Epoch 98
Validation prc-auc = 0.339305
Epoch 99
Loss = 1.9905e-01, PNorm = 41.8080, GNorm = 1.4647, lr_0 = 8.5461e-04
Validation prc-auc = 0.362723
Epoch 100
Validation prc-auc = 0.308210
Epoch 101
Validation prc-auc = 0.309472
Epoch 102
Validation prc-auc = 0.311742
Epoch 103
Validation prc-auc = 0.346417
Epoch 104
Loss = 1.4626e-01, PNorm = 41.8848, GNorm = 1.2433, lr_0 = 8.4730e-04
Validation prc-auc = 0.354198
Epoch 105
Validation prc-auc = 0.361338
Epoch 106
Validation prc-auc = 0.358159
Epoch 107
Validation prc-auc = 0.353278
Epoch 108
Validation prc-auc = 0.351596
Epoch 109
Loss = 2.7813e-01, PNorm = 41.9547, GNorm = 2.7585, lr_0 = 8.4006e-04
Validation prc-auc = 0.356516
Epoch 110
Validation prc-auc = 0.400834
Epoch 111
Validation prc-auc = 0.402471
Epoch 112
Validation prc-auc = 0.401200
Epoch 113
Validation prc-auc = 0.367995
Epoch 114
Loss = 2.0418e-01, PNorm = 42.0283, GNorm = 1.2539, lr_0 = 8.3288e-04
Validation prc-auc = 0.387381
Epoch 115
Validation prc-auc = 0.380034
Epoch 116
Validation prc-auc = 0.373610
Epoch 117
Validation prc-auc = 0.398308
Epoch 118
Validation prc-auc = 0.411291
Epoch 119
Loss = 1.2241e-01, PNorm = 42.1056, GNorm = 1.2443, lr_0 = 8.2576e-04
Validation prc-auc = 0.414158
Epoch 120
Validation prc-auc = 0.404533
Epoch 121
Validation prc-auc = 0.405122
Epoch 122
Validation prc-auc = 0.414315
Epoch 123
Validation prc-auc = 0.416356
Epoch 124
Loss = 1.5549e-01, PNorm = 42.1755, GNorm = 1.6898, lr_0 = 8.1870e-04
Validation prc-auc = 0.424958
Epoch 125
Validation prc-auc = 0.426243
Epoch 126
Validation prc-auc = 0.417015
Epoch 127
Validation prc-auc = 0.414126
Epoch 128
Validation prc-auc = 0.414423
Epoch 129
Loss = 1.3687e-01, PNorm = 42.2351, GNorm = 0.6990, lr_0 = 8.1170e-04
Validation prc-auc = 0.380275
Epoch 130
Validation prc-auc = 0.404328
Epoch 131
Validation prc-auc = 0.383853
Epoch 132
Validation prc-auc = 0.371243
Epoch 133
Validation prc-auc = 0.388533
Epoch 134
Loss = 1.7593e-01, PNorm = 42.2991, GNorm = 1.2565, lr_0 = 8.0476e-04
Validation prc-auc = 0.450535
Epoch 135
Validation prc-auc = 0.451732
Epoch 136
Validation prc-auc = 0.454247
Epoch 137
Validation prc-auc = 0.439496
Epoch 138
Validation prc-auc = 0.412989
Epoch 139
Loss = 2.1591e-01, PNorm = 42.3635, GNorm = 1.5199, lr_0 = 7.9788e-04
Validation prc-auc = 0.404297
Epoch 140
Validation prc-auc = 0.385184
Epoch 141
Validation prc-auc = 0.331074
Epoch 142
Validation prc-auc = 0.323216
Epoch 143
Validation prc-auc = 0.319108
Epoch 144
Loss = 1.6369e-01, PNorm = 42.4218, GNorm = 1.8367, lr_0 = 7.9106e-04
Validation prc-auc = 0.330057
Epoch 145
Validation prc-auc = 0.350324
Epoch 146
Validation prc-auc = 0.366443
Epoch 147
Validation prc-auc = 0.406036
Epoch 148
Validation prc-auc = 0.427438
Epoch 149
Loss = 1.3098e-01, PNorm = 42.4909, GNorm = 1.2837, lr_0 = 7.8430e-04
Validation prc-auc = 0.426315
Epoch 150
Validation prc-auc = 0.403849
Epoch 151
Validation prc-auc = 0.385752
Epoch 152
Validation prc-auc = 0.392141
Epoch 153
Validation prc-auc = 0.377092
Epoch 154
Loss = 1.3231e-01, PNorm = 42.5649, GNorm = 1.8506, lr_0 = 7.7759e-04
Validation prc-auc = 0.372405
Epoch 155
Validation prc-auc = 0.389471
Epoch 156
Validation prc-auc = 0.426822
Epoch 157
Validation prc-auc = 0.450261
Epoch 158
Validation prc-auc = 0.422947
Epoch 159
Loss = 1.3635e-01, PNorm = 42.6303, GNorm = 1.1954, lr_0 = 7.7095e-04
Validation prc-auc = 0.406227
Epoch 160
Validation prc-auc = 0.373335
Epoch 161
Validation prc-auc = 0.315728
Epoch 162
Validation prc-auc = 0.298496
Epoch 163
Validation prc-auc = 0.284745
Epoch 164
Loss = 1.2976e-01, PNorm = 42.6951, GNorm = 1.3394, lr_0 = 7.6436e-04
Validation prc-auc = 0.275917
Epoch 165
Validation prc-auc = 0.285373
Epoch 166
Validation prc-auc = 0.297954
Epoch 167
Validation prc-auc = 0.335388
Epoch 168
Validation prc-auc = 0.403923
Epoch 169
Loss = 1.4685e-01, PNorm = 42.7568, GNorm = 0.9281, lr_0 = 7.5782e-04
Validation prc-auc = 0.411281
Epoch 170
Validation prc-auc = 0.425055
Epoch 171
Validation prc-auc = 0.430753
Epoch 172
Validation prc-auc = 0.418448
Epoch 173
Validation prc-auc = 0.415404
Epoch 174
Loss = 1.2211e-01, PNorm = 42.8160, GNorm = 1.0020, lr_0 = 7.5134e-04
Validation prc-auc = 0.393702
Epoch 175
Validation prc-auc = 0.368855
Epoch 176
Validation prc-auc = 0.350445
Epoch 177
Validation prc-auc = 0.341385
Epoch 178
Validation prc-auc = 0.335883
Epoch 179
Loss = 8.3221e-02, PNorm = 42.8706, GNorm = 1.0337, lr_0 = 7.4492e-04
Validation prc-auc = 0.338656
Epoch 180
Validation prc-auc = 0.351673
Epoch 181
Validation prc-auc = 0.349391
Epoch 182
Validation prc-auc = 0.337848
Epoch 183
Validation prc-auc = 0.305452
Epoch 184
Loss = 1.5402e-01, PNorm = 42.9276, GNorm = 0.9652, lr_0 = 7.3855e-04
Validation prc-auc = 0.303551
Epoch 185
Validation prc-auc = 0.319825
Epoch 186
Validation prc-auc = 0.358672
Epoch 187
Validation prc-auc = 0.382810
Epoch 188
Validation prc-auc = 0.396776
Epoch 189
Loss = 9.0698e-02, PNorm = 43.0051, GNorm = 0.3139, lr_0 = 7.3224e-04
Validation prc-auc = 0.381186
Epoch 190
Validation prc-auc = 0.390502
Epoch 191
Validation prc-auc = 0.407592
Epoch 192
Validation prc-auc = 0.409791
Epoch 193
Validation prc-auc = 0.415678
Epoch 194
Loss = 6.3286e-02, PNorm = 43.0733, GNorm = 0.9747, lr_0 = 7.2598e-04
Validation prc-auc = 0.419292
Epoch 195
Validation prc-auc = 0.408424
Epoch 196
Validation prc-auc = 0.405613
Epoch 197
Validation prc-auc = 0.375613
Epoch 198
Validation prc-auc = 0.342932
Epoch 199
Loss = 6.8848e-02, PNorm = 43.1269, GNorm = 0.7531, lr_0 = 7.1977e-04
Validation prc-auc = 0.332891
Epoch 200
Validation prc-auc = 0.333125
Epoch 201
Validation prc-auc = 0.340185
Epoch 202
Validation prc-auc = 0.346880
Epoch 203
Validation prc-auc = 0.353698
Epoch 204
Loss = 9.9902e-02, PNorm = 43.1801, GNorm = 0.8796, lr_0 = 7.1362e-04
Validation prc-auc = 0.361344
Epoch 205
Validation prc-auc = 0.363505
Epoch 206
Validation prc-auc = 0.367417
Epoch 207
Validation prc-auc = 0.363733
Epoch 208
Validation prc-auc = 0.365022
Epoch 209
Loss = 7.1032e-02, PNorm = 43.2293, GNorm = 1.8124, lr_0 = 7.0752e-04
Validation prc-auc = 0.368608
Epoch 210
Validation prc-auc = 0.386315
Epoch 211
Validation prc-auc = 0.381347
Epoch 212
Validation prc-auc = 0.334426
Epoch 213
Validation prc-auc = 0.319361
Epoch 214
Loss = 1.8461e-01, PNorm = 43.2764, GNorm = 2.6731, lr_0 = 7.0147e-04
Validation prc-auc = 0.311904
Epoch 215
Validation prc-auc = 0.310263
Epoch 216
Validation prc-auc = 0.303647
Epoch 217
Validation prc-auc = 0.303110
Epoch 218
Validation prc-auc = 0.302604
Epoch 219
Loss = 8.2725e-02, PNorm = 43.3359, GNorm = 1.1064, lr_0 = 6.9548e-04
Validation prc-auc = 0.300355
Epoch 220
Validation prc-auc = 0.304157
Epoch 221
Validation prc-auc = 0.318887
Epoch 222
Validation prc-auc = 0.333604
Epoch 223
Validation prc-auc = 0.346072
Epoch 224
Loss = 7.5796e-02, PNorm = 43.4016, GNorm = 0.9223, lr_0 = 6.8953e-04
Validation prc-auc = 0.341491
Epoch 225
Validation prc-auc = 0.333164
Epoch 226
Validation prc-auc = 0.339113
Epoch 227
Validation prc-auc = 0.359351
Epoch 228
Validation prc-auc = 0.349089
Epoch 229
Loss = 8.6279e-02, PNorm = 43.4567, GNorm = 0.4258, lr_0 = 6.8364e-04
Validation prc-auc = 0.332964
Epoch 230
Validation prc-auc = 0.320331
Epoch 231
Validation prc-auc = 0.322671
Epoch 232
Validation prc-auc = 0.329185
Epoch 233
Validation prc-auc = 0.347799
Epoch 234
Loss = 1.3876e-01, PNorm = 43.5038, GNorm = 1.5590, lr_0 = 6.7779e-04
Validation prc-auc = 0.348684
Epoch 235
Validation prc-auc = 0.378462
Epoch 236
Validation prc-auc = 0.382312
Epoch 237
Validation prc-auc = 0.382579
Epoch 238
Validation prc-auc = 0.382574
Epoch 239
Loss = 7.2541e-02, PNorm = 43.5518, GNorm = 0.9756, lr_0 = 6.7200e-04
Validation prc-auc = 0.373693
Epoch 240
Validation prc-auc = 0.352199
Epoch 241
Validation prc-auc = 0.339555
Epoch 242
Validation prc-auc = 0.318075
Epoch 243
Validation prc-auc = 0.311387
Epoch 244
Loss = 7.8593e-02, PNorm = 43.6014, GNorm = 1.4841, lr_0 = 6.6625e-04
Validation prc-auc = 0.304021
Epoch 245
Validation prc-auc = 0.292602
Epoch 246
Validation prc-auc = 0.290799
Epoch 247
Validation prc-auc = 0.290850
Epoch 248
Validation prc-auc = 0.293649
Epoch 249
Loss = 8.2567e-02, PNorm = 43.6478, GNorm = 1.8436, lr_0 = 6.6056e-04
Validation prc-auc = 0.312653
Epoch 250
Validation prc-auc = 0.317545
Epoch 251
Validation prc-auc = 0.326574
Epoch 252
Validation prc-auc = 0.338734
Epoch 253
Validation prc-auc = 0.342434
Epoch 254
Loss = 4.2885e-02, PNorm = 43.6884, GNorm = 0.5990, lr_0 = 6.5491e-04
Validation prc-auc = 0.324615
Epoch 255
Validation prc-auc = 0.318895
Epoch 256
Validation prc-auc = 0.310969
Epoch 257
Validation prc-auc = 0.310268
Epoch 258
Validation prc-auc = 0.307072
Epoch 259
Loss = 4.5999e-02, PNorm = 43.7301, GNorm = 0.7071, lr_0 = 6.4931e-04
Validation prc-auc = 0.310675
Epoch 260
Validation prc-auc = 0.316120
Epoch 261
Validation prc-auc = 0.320247
Epoch 262
Validation prc-auc = 0.329897
Epoch 263
Validation prc-auc = 0.337505
Epoch 264
Loss = 8.5406e-02, PNorm = 43.7656, GNorm = 1.8812, lr_0 = 6.4376e-04
Validation prc-auc = 0.342138
Epoch 265
Validation prc-auc = 0.347713
Epoch 266
Validation prc-auc = 0.351694
Epoch 267
Validation prc-auc = 0.358248
Epoch 268
Validation prc-auc = 0.358464
Epoch 269
Loss = 2.1766e-02, PNorm = 43.7955, GNorm = 0.3427, lr_0 = 6.3826e-04
Validation prc-auc = 0.347909
Epoch 270
Validation prc-auc = 0.342562
Epoch 271
Validation prc-auc = 0.339538
Epoch 272
Validation prc-auc = 0.335255
Epoch 273
Validation prc-auc = 0.327413
Epoch 274
Loss = 1.7662e-01, PNorm = 43.8211, GNorm = 2.8080, lr_0 = 6.3280e-04
Validation prc-auc = 0.324064
Epoch 275
Validation prc-auc = 0.311869
Epoch 276
Validation prc-auc = 0.306418
Epoch 277
Validation prc-auc = 0.302407
Epoch 278
Validation prc-auc = 0.301276
Epoch 279
Loss = 4.9027e-02, PNorm = 43.8485, GNorm = 0.2777, lr_0 = 6.2739e-04
Validation prc-auc = 0.302721
Epoch 280
Validation prc-auc = 0.302598
Epoch 281
Validation prc-auc = 0.305813
Epoch 282
Validation prc-auc = 0.309321
Epoch 283
Validation prc-auc = 0.319143
Epoch 284
Loss = 8.8866e-02, PNorm = 43.8847, GNorm = 0.9603, lr_0 = 6.2203e-04
Validation prc-auc = 0.321734
Epoch 285
Validation prc-auc = 0.343600
Epoch 286
Validation prc-auc = 0.342796
Epoch 287
Validation prc-auc = 0.344437
Epoch 288
Validation prc-auc = 0.343455
Epoch 289
Loss = 8.9113e-02, PNorm = 43.9266, GNorm = 1.1750, lr_0 = 6.1671e-04
Validation prc-auc = 0.347695
Epoch 290
Validation prc-auc = 0.354515
Epoch 291
Validation prc-auc = 0.345629
Epoch 292
Validation prc-auc = 0.341559
Epoch 293
Validation prc-auc = 0.326547
Epoch 294
Loss = 2.3760e-02, PNorm = 43.9607, GNorm = 0.4199, lr_0 = 6.1144e-04
Validation prc-auc = 0.316716
Epoch 295
Validation prc-auc = 0.318207
Epoch 296
Validation prc-auc = 0.330263
Epoch 297
Validation prc-auc = 0.336078
Epoch 298
Validation prc-auc = 0.341520
Epoch 299
Loss = 5.5772e-02, PNorm = 43.9939, GNorm = 1.5600, lr_0 = 6.0621e-04
Validation prc-auc = 0.342389
Model 0 best validation prc-auc = 0.454247 on epoch 136
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test prc-auc = 0.484414
Ensemble test prc-auc = 0.484414
Fold 7
Splitting data with seed 7
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=1728, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=300, out_features=300, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 873,601
Moving model to cuda
Epoch 0
Validation prc-auc = 0.103826
Epoch 1
Validation prc-auc = 0.209197
Epoch 2
Validation prc-auc = 0.363660
Epoch 3
Validation prc-auc = 0.391108
Epoch 4
Loss = 6.2948e-01, PNorm = 40.7422, GNorm = 5.1991, lr_0 = 6.5000e-04
Validation prc-auc = 0.371311
Epoch 5
Validation prc-auc = 0.336201
Epoch 6
Validation prc-auc = 0.332175
Epoch 7
Validation prc-auc = 0.348615
Epoch 8
Validation prc-auc = 0.379567
Epoch 9
Loss = 5.4370e-01, PNorm = 40.8006, GNorm = 2.5234, lr_0 = 9.9743e-04
Validation prc-auc = 0.349674
Epoch 10
Validation prc-auc = 0.338217
Epoch 11
Validation prc-auc = 0.285258
Epoch 12
Validation prc-auc = 0.328291
Epoch 13
Validation prc-auc = 0.326833
Epoch 14
Loss = 5.0628e-01, PNorm = 40.8682, GNorm = 2.0059, lr_0 = 9.8890e-04
Validation prc-auc = 0.356973
Epoch 15
Validation prc-auc = 0.354099
Epoch 16
Validation prc-auc = 0.364778
Epoch 17
Validation prc-auc = 0.360657
Epoch 18
Validation prc-auc = 0.368803
Epoch 19
Loss = 5.5574e-01, PNorm = 40.9233, GNorm = 2.9246, lr_0 = 9.8045e-04
Validation prc-auc = 0.383548
Epoch 20
Validation prc-auc = 0.401997
Epoch 21
Validation prc-auc = 0.423792
Epoch 22
Validation prc-auc = 0.426187
Epoch 23
Validation prc-auc = 0.425495
Epoch 24
Loss = 5.0576e-01, PNorm = 40.9725, GNorm = 1.6887, lr_0 = 9.7207e-04
Validation prc-auc = 0.431285
Epoch 25
Validation prc-auc = 0.431556
Epoch 26
Validation prc-auc = 0.432736
Epoch 27
Validation prc-auc = 0.427335
Epoch 28
Validation prc-auc = 0.414255
Epoch 29
Loss = 4.4811e-01, PNorm = 41.0155, GNorm = 1.6753, lr_0 = 9.6376e-04
Validation prc-auc = 0.400761
Epoch 30
Validation prc-auc = 0.381801
Epoch 31
Validation prc-auc = 0.366961
Epoch 32
Validation prc-auc = 0.375690
Epoch 33
Validation prc-auc = 0.377225
Epoch 34
Loss = 4.0721e-01, PNorm = 41.0545, GNorm = 1.5899, lr_0 = 9.5552e-04
Validation prc-auc = 0.391700
Epoch 35
Validation prc-auc = 0.394530
Epoch 36
Validation prc-auc = 0.373585
Epoch 37
Validation prc-auc = 0.366050
Epoch 38
Validation prc-auc = 0.385002
Epoch 39
Loss = 3.8996e-01, PNorm = 41.0993, GNorm = 1.4368, lr_0 = 9.4735e-04
Validation prc-auc = 0.391244
Epoch 40
Validation prc-auc = 0.396183
Epoch 41
Validation prc-auc = 0.409219
Epoch 42
Validation prc-auc = 0.405509
Epoch 43
Validation prc-auc = 0.402993
Epoch 44
Loss = 3.7905e-01, PNorm = 41.1481, GNorm = 1.6415, lr_0 = 9.3925e-04
Validation prc-auc = 0.389699
Epoch 45
Validation prc-auc = 0.381238
Epoch 46
Validation prc-auc = 0.367666
Epoch 47
Validation prc-auc = 0.364743
Epoch 48
Validation prc-auc = 0.368078
Epoch 49
Loss = 4.2546e-01, PNorm = 41.1983, GNorm = 2.2513, lr_0 = 9.3122e-04
Validation prc-auc = 0.370164
Epoch 50
Validation prc-auc = 0.378776
Epoch 51
Validation prc-auc = 0.398560
Epoch 52
Validation prc-auc = 0.407382
Epoch 53
Validation prc-auc = 0.402335
Epoch 54
Loss = 3.9629e-01, PNorm = 41.2480, GNorm = 1.4761, lr_0 = 9.2326e-04
Validation prc-auc = 0.401651
Epoch 55
Validation prc-auc = 0.394781
Epoch 56
Validation prc-auc = 0.389530
Epoch 57
Validation prc-auc = 0.376921
Epoch 58
Validation prc-auc = 0.371081
Epoch 59
Loss = 4.4281e-01, PNorm = 41.3052, GNorm = 2.2607, lr_0 = 9.1537e-04
Validation prc-auc = 0.353798
Epoch 60
Validation prc-auc = 0.324090
Epoch 61
Validation prc-auc = 0.294288
Epoch 62
Validation prc-auc = 0.276370
Epoch 63
Validation prc-auc = 0.309824
Epoch 64
Loss = 2.0868e-01, PNorm = 41.3710, GNorm = 1.4581, lr_0 = 9.0754e-04
Validation prc-auc = 0.367784
Epoch 65
Validation prc-auc = 0.385603
Epoch 66
Validation prc-auc = 0.387142
Epoch 67
Validation prc-auc = 0.362499
Epoch 68
Validation prc-auc = 0.336790
Epoch 69
Loss = 3.3972e-01, PNorm = 41.4318, GNorm = 1.8761, lr_0 = 8.9978e-04
Validation prc-auc = 0.323308
Epoch 70
Validation prc-auc = 0.326913
Epoch 71
Validation prc-auc = 0.294929
Epoch 72
Validation prc-auc = 0.282391
Epoch 73
Validation prc-auc = 0.305091
Epoch 74
Loss = 2.8307e-01, PNorm = 41.5006, GNorm = 1.5391, lr_0 = 8.9209e-04
Validation prc-auc = 0.297801
Epoch 75
Validation prc-auc = 0.317358
Epoch 76
Validation prc-auc = 0.330937
Epoch 77
Validation prc-auc = 0.341784
Epoch 78
Validation prc-auc = 0.359976
Epoch 79
Loss = 2.9336e-01, PNorm = 41.5677, GNorm = 2.7126, lr_0 = 8.8447e-04
Validation prc-auc = 0.360413
Epoch 80
Validation prc-auc = 0.350566
Epoch 81
Validation prc-auc = 0.338865
Epoch 82
Validation prc-auc = 0.343829
Epoch 83
Validation prc-auc = 0.345197
Epoch 84
Loss = 2.5247e-01, PNorm = 41.6343, GNorm = 2.1061, lr_0 = 8.7691e-04
Validation prc-auc = 0.324767
Epoch 85
Validation prc-auc = 0.303050
Epoch 86
Validation prc-auc = 0.315565
Epoch 87
Validation prc-auc = 0.320954
Epoch 88
Validation prc-auc = 0.318757
Epoch 89
Loss = 2.9478e-01, PNorm = 41.6913, GNorm = 1.3851, lr_0 = 8.6941e-04
Validation prc-auc = 0.313146
Epoch 90
Validation prc-auc = 0.315845
Epoch 91
Validation prc-auc = 0.314715
Epoch 92
Validation prc-auc = 0.314530
Epoch 93
Validation prc-auc = 0.368738
Epoch 94
Loss = 2.6596e-01, PNorm = 41.7553, GNorm = 1.6523, lr_0 = 8.6198e-04
Validation prc-auc = 0.397497
Epoch 95
Validation prc-auc = 0.396239
Epoch 96
Validation prc-auc = 0.370731
Epoch 97
Validation prc-auc = 0.314362
Epoch 98
Validation prc-auc = 0.274546
Epoch 99
Loss = 1.9894e-01, PNorm = 41.8229, GNorm = 3.6097, lr_0 = 8.5461e-04
Validation prc-auc = 0.277773
Epoch 100
Validation prc-auc = 0.283797
Epoch 101
Validation prc-auc = 0.280991
Epoch 102
Validation prc-auc = 0.269477
Epoch 103
Validation prc-auc = 0.263910
Epoch 104
Loss = 1.8983e-01, PNorm = 41.8883, GNorm = 1.6995, lr_0 = 8.4730e-04
Validation prc-auc = 0.268116
Epoch 105
Validation prc-auc = 0.277411
Epoch 106
Validation prc-auc = 0.306351
Epoch 107
Validation prc-auc = 0.330554
Epoch 108
Validation prc-auc = 0.367504
Epoch 109
Loss = 1.4423e-01, PNorm = 41.9560, GNorm = 1.0925, lr_0 = 8.4006e-04
Validation prc-auc = 0.368051
Epoch 110
Validation prc-auc = 0.361947
Epoch 111
Validation prc-auc = 0.353884
Epoch 112
Validation prc-auc = 0.330649
Epoch 113
Validation prc-auc = 0.327498
Epoch 114
Loss = 2.0665e-01, PNorm = 42.0213, GNorm = 1.4221, lr_0 = 8.3288e-04
Validation prc-auc = 0.306193
Epoch 115
Validation prc-auc = 0.292978
Epoch 116
Validation prc-auc = 0.297860
Epoch 117
Validation prc-auc = 0.292082
Epoch 118
Validation prc-auc = 0.288404
Epoch 119
Loss = 1.7123e-01, PNorm = 42.0778, GNorm = 1.7736, lr_0 = 8.2576e-04
Validation prc-auc = 0.291223
Epoch 120
Validation prc-auc = 0.310367
Epoch 121
Validation prc-auc = 0.330882
Epoch 122
Validation prc-auc = 0.339324
Epoch 123
Validation prc-auc = 0.331931
Epoch 124
Loss = 1.2791e-01, PNorm = 42.1381, GNorm = 0.9308, lr_0 = 8.1870e-04
Validation prc-auc = 0.322667
Epoch 125
Validation prc-auc = 0.312767
Epoch 126
Validation prc-auc = 0.311596
Epoch 127
Validation prc-auc = 0.314248
Epoch 128
Validation prc-auc = 0.306904
Epoch 129
Loss = 1.2226e-01, PNorm = 42.1943, GNorm = 1.0924, lr_0 = 8.1170e-04
Validation prc-auc = 0.291503
Epoch 130
Validation prc-auc = 0.289016
Epoch 131
Validation prc-auc = 0.296636
Epoch 132
Validation prc-auc = 0.311641
Epoch 133
Validation prc-auc = 0.319818
Epoch 134
Loss = 2.6118e-01, PNorm = 42.2637, GNorm = 2.5468, lr_0 = 8.0476e-04
Validation prc-auc = 0.301772
Epoch 135
Validation prc-auc = 0.280519
Epoch 136
Validation prc-auc = 0.273185
Epoch 137
Validation prc-auc = 0.274553
Epoch 138
Validation prc-auc = 0.286114
Epoch 139
Loss = 2.1538e-01, PNorm = 42.3266, GNorm = 1.5635, lr_0 = 7.9788e-04
Validation prc-auc = 0.308907
Epoch 140
Validation prc-auc = 0.317161
Epoch 141
Validation prc-auc = 0.320061
Epoch 142
Validation prc-auc = 0.321023
Epoch 143
Validation prc-auc = 0.323831
Epoch 144
Loss = 1.6819e-01, PNorm = 42.3884, GNorm = 1.1841, lr_0 = 7.9106e-04
Validation prc-auc = 0.333016
Epoch 145
Validation prc-auc = 0.337700
Epoch 146
Validation prc-auc = 0.329268
Epoch 147
Validation prc-auc = 0.327410
Epoch 148
Validation prc-auc = 0.328030
Epoch 149
Loss = 1.0727e-01, PNorm = 42.4479, GNorm = 1.0450, lr_0 = 7.8430e-04
Validation prc-auc = 0.310209
Epoch 150
Validation prc-auc = 0.293350
Epoch 151
Validation prc-auc = 0.291243
Epoch 152
Validation prc-auc = 0.298294
Epoch 153
Validation prc-auc = 0.322822
Epoch 154
Loss = 2.2756e-01, PNorm = 42.5064, GNorm = 1.5073, lr_0 = 7.7759e-04
Validation prc-auc = 0.327845
Epoch 155
Validation prc-auc = 0.344494
Epoch 156
Validation prc-auc = 0.349501
Epoch 157
Validation prc-auc = 0.330498
Epoch 158
Validation prc-auc = 0.323327
Epoch 159
Loss = 2.0385e-01, PNorm = 42.5754, GNorm = 1.9946, lr_0 = 7.7095e-04
Validation prc-auc = 0.337729
Epoch 160
Validation prc-auc = 0.347703
Epoch 161
Validation prc-auc = 0.369757
Epoch 162
Validation prc-auc = 0.386616
Epoch 163
Validation prc-auc = 0.382440
Epoch 164
Loss = 2.0629e-01, PNorm = 42.6414, GNorm = 1.6111, lr_0 = 7.6436e-04
Validation prc-auc = 0.367839
Epoch 165
Validation prc-auc = 0.360137
Epoch 166
Validation prc-auc = 0.340947
Epoch 167
Validation prc-auc = 0.322840
Epoch 168
Validation prc-auc = 0.316605
Epoch 169
Loss = 1.3388e-01, PNorm = 42.6991, GNorm = 1.4698, lr_0 = 7.5782e-04
Validation prc-auc = 0.317249
Epoch 170
Validation prc-auc = 0.333201
Epoch 171
Validation prc-auc = 0.335295
Epoch 172
Validation prc-auc = 0.327175
Epoch 173
Validation prc-auc = 0.323304
Epoch 174
Loss = 1.0751e-01, PNorm = 42.7569, GNorm = 0.6732, lr_0 = 7.5134e-04
Validation prc-auc = 0.323643
Epoch 175
Validation prc-auc = 0.326988
Epoch 176
Validation prc-auc = 0.326611
Epoch 177
Validation prc-auc = 0.345613
Epoch 178
Validation prc-auc = 0.333047
Epoch 179
Loss = 1.1521e-01, PNorm = 42.8140, GNorm = 1.5582, lr_0 = 7.4492e-04
Validation prc-auc = 0.312754
Epoch 180
Validation prc-auc = 0.303749
Epoch 181
Validation prc-auc = 0.292059
Epoch 182
Validation prc-auc = 0.283353
Epoch 183
Validation prc-auc = 0.277048
Epoch 184
Loss = 1.5206e-01, PNorm = 42.8738, GNorm = 1.6595, lr_0 = 7.3855e-04
Validation prc-auc = 0.283334
Epoch 185
Validation prc-auc = 0.298285
Epoch 186
Validation prc-auc = 0.326557
Epoch 187
Validation prc-auc = 0.328827
Epoch 188
Validation prc-auc = 0.328843
Epoch 189
Loss = 1.7186e-01, PNorm = 42.9303, GNorm = 1.3895, lr_0 = 7.3224e-04
Validation prc-auc = 0.326843
Epoch 190
Validation prc-auc = 0.311580
Epoch 191
Validation prc-auc = 0.303203
Epoch 192
Validation prc-auc = 0.301272
Epoch 193
Validation prc-auc = 0.305756
Epoch 194
Loss = 8.3684e-02, PNorm = 42.9834, GNorm = 1.4901, lr_0 = 7.2598e-04
Validation prc-auc = 0.304576
Epoch 195
Validation prc-auc = 0.302846
Epoch 196
Validation prc-auc = 0.296206
Epoch 197
Validation prc-auc = 0.295218
Epoch 198
Validation prc-auc = 0.289794
Epoch 199
Loss = 1.0134e-01, PNorm = 43.0269, GNorm = 1.3763, lr_0 = 7.1977e-04
Validation prc-auc = 0.289047
Epoch 200
Validation prc-auc = 0.289081
Epoch 201
Validation prc-auc = 0.291482
Epoch 202
Validation prc-auc = 0.284937
Epoch 203
Validation prc-auc = 0.286213
Epoch 204
Loss = 9.2391e-02, PNorm = 43.0697, GNorm = 1.5277, lr_0 = 7.1362e-04
Validation prc-auc = 0.286488
Epoch 205
Validation prc-auc = 0.286039
Epoch 206
Validation prc-auc = 0.283262
Epoch 207
Validation prc-auc = 0.281180
Epoch 208
Validation prc-auc = 0.286835
Epoch 209
Loss = 8.2520e-02, PNorm = 43.1206, GNorm = 0.8200, lr_0 = 7.0752e-04
Validation prc-auc = 0.298267
Epoch 210
Validation prc-auc = 0.303539
Epoch 211
Validation prc-auc = 0.307840
Epoch 212
Validation prc-auc = 0.302194
Epoch 213
Validation prc-auc = 0.304078
Epoch 214
Loss = 1.0753e-01, PNorm = 43.1683, GNorm = 0.9509, lr_0 = 7.0147e-04
Validation prc-auc = 0.297530
Epoch 215
Validation prc-auc = 0.265535
Epoch 216
Validation prc-auc = 0.264714
Epoch 217
Validation prc-auc = 0.264515
Epoch 218
Validation prc-auc = 0.265550
Epoch 219
Loss = 1.5725e-01, PNorm = 43.2267, GNorm = 1.4752, lr_0 = 6.9548e-04
Validation prc-auc = 0.273189
Epoch 220
Validation prc-auc = 0.308579
Epoch 221
Validation prc-auc = 0.314427
Epoch 222
Validation prc-auc = 0.316920
Epoch 223
Validation prc-auc = 0.312577
Epoch 224
Loss = 1.1504e-01, PNorm = 43.2876, GNorm = 1.6741, lr_0 = 6.8953e-04
Validation prc-auc = 0.310113
Epoch 225
Validation prc-auc = 0.310671
Epoch 226
Validation prc-auc = 0.322379
Epoch 227
Validation prc-auc = 0.324664
Epoch 228
Validation prc-auc = 0.320178
Epoch 229
Loss = 6.4022e-02, PNorm = 43.3409, GNorm = 1.0619, lr_0 = 6.8364e-04
Validation prc-auc = 0.310631
Epoch 230
Validation prc-auc = 0.306687
Epoch 231
Validation prc-auc = 0.299401
Epoch 232
Validation prc-auc = 0.299076
Epoch 233
Validation prc-auc = 0.302921
Epoch 234
Loss = 8.1979e-02, PNorm = 43.3892, GNorm = 1.7529, lr_0 = 6.7779e-04
Validation prc-auc = 0.305906
Epoch 235
Validation prc-auc = 0.309521
Epoch 236
Validation prc-auc = 0.314757
Epoch 237
Validation prc-auc = 0.312215
Epoch 238
Validation prc-auc = 0.307132
Epoch 239
Loss = 1.1069e-01, PNorm = 43.4359, GNorm = 1.8978, lr_0 = 6.7200e-04
Validation prc-auc = 0.301055
Epoch 240
Validation prc-auc = 0.297293
Epoch 241
Validation prc-auc = 0.294241
Epoch 242
Validation prc-auc = 0.294955
Epoch 243
Validation prc-auc = 0.301777
Epoch 244
Loss = 5.0050e-02, PNorm = 43.4913, GNorm = 1.1314, lr_0 = 6.6625e-04
Validation prc-auc = 0.305734
Epoch 245
Validation prc-auc = 0.316157
Epoch 246
Validation prc-auc = 0.324696
Epoch 247
Validation prc-auc = 0.324445
Epoch 248
Validation prc-auc = 0.325222
Epoch 249
Loss = 5.7313e-02, PNorm = 43.5393, GNorm = 0.3556, lr_0 = 6.6056e-04
Validation prc-auc = 0.320083
Epoch 250
Validation prc-auc = 0.317208
Epoch 251
Validation prc-auc = 0.311037
Epoch 252
Validation prc-auc = 0.301392
Epoch 253
Validation prc-auc = 0.294694
Epoch 254
Loss = 4.9141e-02, PNorm = 43.5764, GNorm = 1.0707, lr_0 = 6.5491e-04
Validation prc-auc = 0.296540
Epoch 255
Validation prc-auc = 0.295896
Epoch 256
Validation prc-auc = 0.294187
Epoch 257
Validation prc-auc = 0.284593
Epoch 258
Validation prc-auc = 0.282411
Epoch 259
Loss = 7.9694e-02, PNorm = 43.6121, GNorm = 0.8471, lr_0 = 6.4931e-04
Validation prc-auc = 0.288681
Epoch 260
Validation prc-auc = 0.293549
Epoch 261
Validation prc-auc = 0.307541
Epoch 262
Validation prc-auc = 0.308659
Epoch 263
Validation prc-auc = 0.312640
Epoch 264
Loss = 4.7066e-02, PNorm = 43.6589, GNorm = 0.6440, lr_0 = 6.4376e-04
Validation prc-auc = 0.322715
Epoch 265
Validation prc-auc = 0.328316
Epoch 266
Validation prc-auc = 0.333334
Epoch 267
Validation prc-auc = 0.338261
Epoch 268
Validation prc-auc = 0.335077
Epoch 269
Loss = 2.9083e-02, PNorm = 43.7049, GNorm = 0.3873, lr_0 = 6.3826e-04
Validation prc-auc = 0.330169
Epoch 270
Validation prc-auc = 0.323586
Epoch 271
Validation prc-auc = 0.326168
Epoch 272
Validation prc-auc = 0.325539
Epoch 273
Validation prc-auc = 0.324225
Epoch 274
Loss = 5.3782e-02, PNorm = 43.7544, GNorm = 1.2290, lr_0 = 6.3280e-04
Validation prc-auc = 0.325088
Epoch 275
Validation prc-auc = 0.345821
Epoch 276
Validation prc-auc = 0.352665
Epoch 277
Validation prc-auc = 0.369753
Epoch 278
Validation prc-auc = 0.354743
Epoch 279
Loss = 9.5714e-02, PNorm = 43.7998, GNorm = 1.6577, lr_0 = 6.2739e-04
Validation prc-auc = 0.352052
Epoch 280
Validation prc-auc = 0.340657
Epoch 281
Validation prc-auc = 0.321994
Epoch 282
Validation prc-auc = 0.315024
Epoch 283
Validation prc-auc = 0.310998
Epoch 284
Loss = 4.3614e-02, PNorm = 43.8485, GNorm = 0.6308, lr_0 = 6.2203e-04
Validation prc-auc = 0.309270
Epoch 285
Validation prc-auc = 0.315561
Epoch 286
Validation prc-auc = 0.315010
Epoch 287
Validation prc-auc = 0.325553
Epoch 288
Validation prc-auc = 0.334641
Epoch 289
Loss = 1.2040e-01, PNorm = 43.8925, GNorm = 1.4367, lr_0 = 6.1671e-04
Validation prc-auc = 0.367789
Epoch 290
Validation prc-auc = 0.415224
Epoch 291
Validation prc-auc = 0.381134
Epoch 292
Validation prc-auc = 0.333452
Epoch 293
Validation prc-auc = 0.315735
Epoch 294
Loss = 4.4765e-02, PNorm = 43.9270, GNorm = 0.3050, lr_0 = 6.1144e-04
Validation prc-auc = 0.300686
Epoch 295
Validation prc-auc = 0.291357
Epoch 296
Validation prc-auc = 0.286399
Epoch 297
Validation prc-auc = 0.282526
Epoch 298
Validation prc-auc = 0.280429
Epoch 299
Loss = 2.7119e-02, PNorm = 43.9640, GNorm = 0.4250, lr_0 = 6.0621e-04
Validation prc-auc = 0.277549
Model 0 best validation prc-auc = 0.432736 on epoch 26
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test prc-auc = 0.469799
Ensemble test prc-auc = 0.469799
Fold 8
Splitting data with seed 8
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=1728, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=300, out_features=300, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 873,601
Moving model to cuda
Epoch 0
Validation prc-auc = 0.101485
Epoch 1
Validation prc-auc = 0.180901
Epoch 2
Validation prc-auc = 0.326747
Epoch 3
Validation prc-auc = 0.386182
Epoch 4
Loss = 6.5633e-01, PNorm = 40.7402, GNorm = 2.9232, lr_0 = 6.5000e-04
Validation prc-auc = 0.440543
Epoch 5
Validation prc-auc = 0.456540
Epoch 6
Validation prc-auc = 0.470326
Epoch 7
Validation prc-auc = 0.442466
Epoch 8
Validation prc-auc = 0.435518
Epoch 9
Loss = 5.7857e-01, PNorm = 40.7968, GNorm = 2.1150, lr_0 = 9.9743e-04
Validation prc-auc = 0.422881
Epoch 10
Validation prc-auc = 0.406288
Epoch 11
Validation prc-auc = 0.398381
Epoch 12
Validation prc-auc = 0.376581
Epoch 13
Validation prc-auc = 0.369053
Epoch 14
Loss = 5.0533e-01, PNorm = 40.8646, GNorm = 2.5502, lr_0 = 9.8890e-04
Validation prc-auc = 0.362988
Epoch 15
Validation prc-auc = 0.362745
Epoch 16
Validation prc-auc = 0.357871
Epoch 17
Validation prc-auc = 0.352596
Epoch 18
Validation prc-auc = 0.347779
Epoch 19
Loss = 4.6107e-01, PNorm = 40.9231, GNorm = 1.9209, lr_0 = 9.8045e-04
Validation prc-auc = 0.347706
Epoch 20
Validation prc-auc = 0.344108
Epoch 21
Validation prc-auc = 0.333662
Epoch 22
Validation prc-auc = 0.325988
Epoch 23
Validation prc-auc = 0.246509
Epoch 24
Loss = 5.0811e-01, PNorm = 40.9696, GNorm = 2.0711, lr_0 = 9.7207e-04
Validation prc-auc = 0.246484
Epoch 25
Validation prc-auc = 0.287966
Epoch 26
Validation prc-auc = 0.290634
Epoch 27
Validation prc-auc = 0.305368
Epoch 28
Validation prc-auc = 0.316762
Epoch 29
Loss = 4.3482e-01, PNorm = 41.0156, GNorm = 1.9199, lr_0 = 9.6376e-04
Validation prc-auc = 0.335111
Epoch 30
Validation prc-auc = 0.373834
Epoch 31
Validation prc-auc = 0.382282
Epoch 32
Validation prc-auc = 0.369527
Epoch 33
Validation prc-auc = 0.346635
Epoch 34
Loss = 6.2890e-01, PNorm = 41.0614, GNorm = 2.0657, lr_0 = 9.5552e-04
Validation prc-auc = 0.335658
Epoch 35
Validation prc-auc = 0.315212
Epoch 36
Validation prc-auc = 0.309118
Epoch 37
Validation prc-auc = 0.315874
Epoch 38
Validation prc-auc = 0.351926
Epoch 39
Loss = 4.1997e-01, PNorm = 41.1108, GNorm = 1.4077, lr_0 = 9.4735e-04
Validation prc-auc = 0.360368
Epoch 40
Validation prc-auc = 0.371755
Epoch 41
Validation prc-auc = 0.389335
Epoch 42
Validation prc-auc = 0.393897
Epoch 43
Validation prc-auc = 0.391231
Epoch 44
Loss = 4.0706e-01, PNorm = 41.1629, GNorm = 1.5577, lr_0 = 9.3925e-04
Validation prc-auc = 0.386498
Epoch 45
Validation prc-auc = 0.378569
Epoch 46
Validation prc-auc = 0.360330
Epoch 47
Validation prc-auc = 0.343069
Epoch 48
Validation prc-auc = 0.331520
Epoch 49
Loss = 4.2828e-01, PNorm = 41.2155, GNorm = 2.0238, lr_0 = 9.3122e-04
Validation prc-auc = 0.340453
Epoch 50
Validation prc-auc = 0.352427
Epoch 51
Validation prc-auc = 0.318231
Epoch 52
Validation prc-auc = 0.314761
Epoch 53
Validation prc-auc = 0.333891
Epoch 54
Loss = 2.6990e-01, PNorm = 41.2673, GNorm = 1.5059, lr_0 = 9.2326e-04
Validation prc-auc = 0.349606
Epoch 55
Validation prc-auc = 0.321454
Epoch 56
Validation prc-auc = 0.320478
Epoch 57
Validation prc-auc = 0.312406
Epoch 58
Validation prc-auc = 0.300992
Epoch 59
Loss = 3.0400e-01, PNorm = 41.3280, GNorm = 1.3580, lr_0 = 9.1537e-04
Validation prc-auc = 0.302372
Epoch 60
Validation prc-auc = 0.304837
Epoch 61
Validation prc-auc = 0.312442
Epoch 62
Validation prc-auc = 0.334729
Epoch 63
Validation prc-auc = 0.343642
Epoch 64
Loss = 2.8148e-01, PNorm = 41.3873, GNorm = 1.6591, lr_0 = 9.0754e-04
Validation prc-auc = 0.326604
Epoch 65
Validation prc-auc = 0.310241
Epoch 66
Validation prc-auc = 0.312413
Epoch 67
Validation prc-auc = 0.317476
Epoch 68
Validation prc-auc = 0.315045
Epoch 69
Loss = 3.6512e-01, PNorm = 41.4470, GNorm = 4.7423, lr_0 = 8.9978e-04
Validation prc-auc = 0.315428
Epoch 70
Validation prc-auc = 0.330462
Epoch 71
Validation prc-auc = 0.355996
Epoch 72
Validation prc-auc = 0.329669
Epoch 73
Validation prc-auc = 0.320861
Epoch 74
Loss = 3.2138e-01, PNorm = 41.5022, GNorm = 1.8804, lr_0 = 8.9209e-04
Validation prc-auc = 0.302505
Epoch 75
Validation prc-auc = 0.314501
Epoch 76
Validation prc-auc = 0.321544
Epoch 77
Validation prc-auc = 0.323690
Epoch 78
Validation prc-auc = 0.287822
Epoch 79
Loss = 2.7489e-01, PNorm = 41.5656, GNorm = 1.7531, lr_0 = 8.8447e-04
Validation prc-auc = 0.290344
Epoch 80
Validation prc-auc = 0.301207
Epoch 81
Validation prc-auc = 0.314483
Epoch 82
Validation prc-auc = 0.344144
Epoch 83
Validation prc-auc = 0.361541
Epoch 84
Loss = 2.5299e-01, PNorm = 41.6323, GNorm = 1.3608, lr_0 = 8.7691e-04
Validation prc-auc = 0.366624
Epoch 85
Validation prc-auc = 0.362657
Epoch 86
Validation prc-auc = 0.358089
Epoch 87
Validation prc-auc = 0.357565
Epoch 88
Validation prc-auc = 0.334058
Epoch 89
Loss = 3.1535e-01, PNorm = 41.7002, GNorm = 1.6060, lr_0 = 8.6941e-04
Validation prc-auc = 0.337089
Epoch 90
Validation prc-auc = 0.326869
Epoch 91
Validation prc-auc = 0.330858
Epoch 92
Validation prc-auc = 0.334592
Epoch 93
Validation prc-auc = 0.336290
Epoch 94
Loss = 3.3551e-01, PNorm = 41.7665, GNorm = 2.1360, lr_0 = 8.6198e-04
Validation prc-auc = 0.334307
Epoch 95
Validation prc-auc = 0.334898
Epoch 96
Validation prc-auc = 0.321687
Epoch 97
Validation prc-auc = 0.329916
Epoch 98
Validation prc-auc = 0.379719
Epoch 99
Loss = 4.3674e-01, PNorm = 41.8340, GNorm = 5.7089, lr_0 = 8.5461e-04
Validation prc-auc = 0.386369
Epoch 100
Validation prc-auc = 0.395553
Epoch 101
Validation prc-auc = 0.358665
Epoch 102
Validation prc-auc = 0.349333
Epoch 103
Validation prc-auc = 0.344888
Epoch 104
Loss = 1.8922e-01, PNorm = 41.9005, GNorm = 1.8814, lr_0 = 8.4730e-04
Validation prc-auc = 0.331166
Epoch 105
Validation prc-auc = 0.350146
Epoch 106
Validation prc-auc = 0.351190
Epoch 107
Validation prc-auc = 0.353085
Epoch 108
Validation prc-auc = 0.372562
Epoch 109
Loss = 2.0429e-01, PNorm = 41.9569, GNorm = 1.9578, lr_0 = 8.4006e-04
Validation prc-auc = 0.367835
Epoch 110
Validation prc-auc = 0.335329
Epoch 111
Validation prc-auc = 0.337624
Epoch 112
Validation prc-auc = 0.351859
Epoch 113
Validation prc-auc = 0.343698
Epoch 114
Loss = 2.2195e-01, PNorm = 42.0209, GNorm = 1.5919, lr_0 = 8.3288e-04
Validation prc-auc = 0.327667
Epoch 115
Validation prc-auc = 0.332738
Epoch 116
Validation prc-auc = 0.329600
Epoch 117
Validation prc-auc = 0.360705
Epoch 118
Validation prc-auc = 0.383630
Epoch 119
Loss = 2.2671e-01, PNorm = 42.0898, GNorm = 1.9778, lr_0 = 8.2576e-04
Validation prc-auc = 0.381397
Epoch 120
Validation prc-auc = 0.375304
Epoch 121
Validation prc-auc = 0.373492
Epoch 122
Validation prc-auc = 0.350608
Epoch 123
Validation prc-auc = 0.310782
Epoch 124
Loss = 1.6390e-01, PNorm = 42.1564, GNorm = 0.9635, lr_0 = 8.1870e-04
Validation prc-auc = 0.295819
Epoch 125
Validation prc-auc = 0.300642
Epoch 126
Validation prc-auc = 0.330166
Epoch 127
Validation prc-auc = 0.353509
Epoch 128
Validation prc-auc = 0.363307
Epoch 129
Loss = 1.2174e-01, PNorm = 42.2261, GNorm = 1.2319, lr_0 = 8.1170e-04
Validation prc-auc = 0.343385
Epoch 130
Validation prc-auc = 0.360508
Epoch 131
Validation prc-auc = 0.373966
Epoch 132
Validation prc-auc = 0.386287
Epoch 133
Validation prc-auc = 0.392398
Epoch 134
Loss = 1.4730e-01, PNorm = 42.2876, GNorm = 1.3946, lr_0 = 8.0476e-04
Validation prc-auc = 0.387660
Epoch 135
Validation prc-auc = 0.373789
Epoch 136
Validation prc-auc = 0.368544
Epoch 137
Validation prc-auc = 0.382690
Epoch 138
Validation prc-auc = 0.392536
Epoch 139
Loss = 1.3561e-01, PNorm = 42.3421, GNorm = 1.3831, lr_0 = 7.9788e-04
Validation prc-auc = 0.396884
Epoch 140
Validation prc-auc = 0.397458
Epoch 141
Validation prc-auc = 0.385580
Epoch 142
Validation prc-auc = 0.333714
Epoch 143
Validation prc-auc = 0.310465
Epoch 144
Loss = 2.8982e-01, PNorm = 42.4008, GNorm = 2.0760, lr_0 = 7.9106e-04
Validation prc-auc = 0.303825
Epoch 145
Validation prc-auc = 0.302505
Epoch 146
Validation prc-auc = 0.304031
Epoch 147
Validation prc-auc = 0.314983
Epoch 148
Validation prc-auc = 0.318545
Epoch 149
Loss = 1.5770e-01, PNorm = 42.4547, GNorm = 1.5415, lr_0 = 7.8430e-04
Validation prc-auc = 0.318107
Epoch 150
Validation prc-auc = 0.309013
Epoch 151
Validation prc-auc = 0.299557
Epoch 152
Validation prc-auc = 0.312406
Epoch 153
Validation prc-auc = 0.311805
Epoch 154
Loss = 7.3729e-02, PNorm = 42.5099, GNorm = 0.4429, lr_0 = 7.7759e-04
Validation prc-auc = 0.309995
Epoch 155
Validation prc-auc = 0.294926
Epoch 156
Validation prc-auc = 0.296024
Epoch 157
Validation prc-auc = 0.301274
Epoch 158
Validation prc-auc = 0.306546
Epoch 159
Loss = 1.0524e-01, PNorm = 42.5704, GNorm = 1.4859, lr_0 = 7.7095e-04
Validation prc-auc = 0.314714
Epoch 160
Validation prc-auc = 0.315016
Epoch 161
Validation prc-auc = 0.318486
Epoch 162
Validation prc-auc = 0.321696
Epoch 163
Validation prc-auc = 0.330000
Epoch 164
Loss = 1.1363e-01, PNorm = 42.6190, GNorm = 1.4521, lr_0 = 7.6436e-04
Validation prc-auc = 0.338387
Epoch 165
Validation prc-auc = 0.348698
Epoch 166
Validation prc-auc = 0.354232
Epoch 167
Validation prc-auc = 0.367900
Epoch 168
Validation prc-auc = 0.365751
Epoch 169
Loss = 7.5491e-02, PNorm = 42.6674, GNorm = 1.4330, lr_0 = 7.5782e-04
Validation prc-auc = 0.349862
Epoch 170
Validation prc-auc = 0.327093
Epoch 171
Validation prc-auc = 0.322818
Epoch 172
Validation prc-auc = 0.323630
Epoch 173
Validation prc-auc = 0.359885
Epoch 174
Loss = 1.6780e-01, PNorm = 42.7156, GNorm = 0.9725, lr_0 = 7.5134e-04
Validation prc-auc = 0.347336
Epoch 175
Validation prc-auc = 0.329327
Epoch 176
Validation prc-auc = 0.325975
Epoch 177
Validation prc-auc = 0.326759
Epoch 178
Validation prc-auc = 0.338106
Epoch 179
Loss = 8.7797e-02, PNorm = 42.7675, GNorm = 0.8177, lr_0 = 7.4492e-04
Validation prc-auc = 0.331854
Epoch 180
Validation prc-auc = 0.337568
Epoch 181
Validation prc-auc = 0.339403
Epoch 182
Validation prc-auc = 0.327521
Epoch 183
Validation prc-auc = 0.316915
Epoch 184
Loss = 1.6052e-01, PNorm = 42.8223, GNorm = 2.4565, lr_0 = 7.3855e-04
Validation prc-auc = 0.321099
Epoch 185
Validation prc-auc = 0.331892
Epoch 186
Validation prc-auc = 0.331876
Epoch 187
Validation prc-auc = 0.328366
Epoch 188
Validation prc-auc = 0.327803
Epoch 189
Loss = 1.4881e-01, PNorm = 42.8824, GNorm = 1.9276, lr_0 = 7.3224e-04
Validation prc-auc = 0.367896
Epoch 190
Validation prc-auc = 0.405186
Epoch 191
Validation prc-auc = 0.394090
Epoch 192
Validation prc-auc = 0.361026
Epoch 193
Validation prc-auc = 0.342905
Epoch 194
Loss = 8.3865e-02, PNorm = 42.9427, GNorm = 1.2258, lr_0 = 7.2598e-04
Validation prc-auc = 0.317752
Epoch 195
Validation prc-auc = 0.316978
Epoch 196
Validation prc-auc = 0.328315
Epoch 197
Validation prc-auc = 0.326040
Epoch 198
Validation prc-auc = 0.325693
Epoch 199
Loss = 1.8575e-01, PNorm = 42.9983, GNorm = 2.0644, lr_0 = 7.1977e-04
Validation prc-auc = 0.322361
Epoch 200
Validation prc-auc = 0.334659
Epoch 201
Validation prc-auc = 0.331152
Epoch 202
Validation prc-auc = 0.333892
Epoch 203
Validation prc-auc = 0.329363
Epoch 204
Loss = 4.3736e-02, PNorm = 43.0561, GNorm = 0.4067, lr_0 = 7.1362e-04
Validation prc-auc = 0.317667
Epoch 205
Validation prc-auc = 0.316893
Epoch 206
Validation prc-auc = 0.319863
Epoch 207
Validation prc-auc = 0.328684
Epoch 208
Validation prc-auc = 0.344250
Epoch 209
Loss = 6.6644e-02, PNorm = 43.0981, GNorm = 0.9255, lr_0 = 7.0752e-04
Validation prc-auc = 0.338148
Epoch 210
Validation prc-auc = 0.341200
Epoch 211
Validation prc-auc = 0.358966
Epoch 212
Validation prc-auc = 0.366379
Epoch 213
Validation prc-auc = 0.362162
Epoch 214
Loss = 4.5093e-02, PNorm = 43.1415, GNorm = 0.7741, lr_0 = 7.0147e-04
Validation prc-auc = 0.357714
Epoch 215
Validation prc-auc = 0.364510
Epoch 216
Validation prc-auc = 0.366788
Epoch 217
Validation prc-auc = 0.381785
Epoch 218
Validation prc-auc = 0.431879
Epoch 219
Loss = 1.1927e-01, PNorm = 43.1832, GNorm = 2.2844, lr_0 = 6.9548e-04
Validation prc-auc = 0.404819
Epoch 220
Validation prc-auc = 0.366272
Epoch 221
Validation prc-auc = 0.334504
Epoch 222
Validation prc-auc = 0.326688
Epoch 223
Validation prc-auc = 0.325340
Epoch 224
Loss = 4.6841e-02, PNorm = 43.2246, GNorm = 0.5524, lr_0 = 6.8953e-04
Validation prc-auc = 0.322991
Epoch 225
Validation prc-auc = 0.312549
Epoch 226
Validation prc-auc = 0.314436
Epoch 227
Validation prc-auc = 0.309552
Epoch 228
Validation prc-auc = 0.307852
Epoch 229
Loss = 6.5469e-02, PNorm = 43.2754, GNorm = 0.7393, lr_0 = 6.8364e-04
Validation prc-auc = 0.317050
Epoch 230
Validation prc-auc = 0.331715
Epoch 231
Validation prc-auc = 0.335377
Epoch 232
Validation prc-auc = 0.340935
Epoch 233
Validation prc-auc = 0.340726
Epoch 234
Loss = 4.8735e-02, PNorm = 43.3265, GNorm = 1.0526, lr_0 = 6.7779e-04
Validation prc-auc = 0.344917
Epoch 235
Validation prc-auc = 0.354501
Epoch 236
Validation prc-auc = 0.321462
Epoch 237
Validation prc-auc = 0.350627
Epoch 238
Validation prc-auc = 0.339695
Epoch 239
Loss = 1.1677e-01, PNorm = 43.3683, GNorm = 1.7408, lr_0 = 6.7200e-04
Validation prc-auc = 0.347449
Epoch 240
Validation prc-auc = 0.352364
Epoch 241
Validation prc-auc = 0.340878
Epoch 242
Validation prc-auc = 0.334123
Epoch 243
Validation prc-auc = 0.322537
Epoch 244
Loss = 7.4200e-02, PNorm = 43.4201, GNorm = 0.8168, lr_0 = 6.6625e-04
Validation prc-auc = 0.329756
Epoch 245
Validation prc-auc = 0.333291
Epoch 246
Validation prc-auc = 0.367556
Epoch 247
Validation prc-auc = 0.384167
Epoch 248
Validation prc-auc = 0.387230
Epoch 249
Loss = 8.6122e-02, PNorm = 43.4648, GNorm = 1.8777, lr_0 = 6.6056e-04
Validation prc-auc = 0.390264
Epoch 250
Validation prc-auc = 0.391729
Epoch 251
Validation prc-auc = 0.395309
Epoch 252
Validation prc-auc = 0.374337
Epoch 253
Validation prc-auc = 0.349990
Epoch 254
Loss = 9.4080e-02, PNorm = 43.5058, GNorm = 1.0201, lr_0 = 6.5491e-04
Validation prc-auc = 0.338992
Epoch 255
Validation prc-auc = 0.340369
Epoch 256
Validation prc-auc = 0.332829
Epoch 257
Validation prc-auc = 0.327031
Epoch 258
Validation prc-auc = 0.322399
Epoch 259
Loss = 3.3393e-02, PNorm = 43.5482, GNorm = 0.3422, lr_0 = 6.4931e-04
Validation prc-auc = 0.319053
Epoch 260
Validation prc-auc = 0.330023
Epoch 261
Validation prc-auc = 0.347887
Epoch 262
Validation prc-auc = 0.353569
Epoch 263
Validation prc-auc = 0.364611
Epoch 264
Loss = 4.6152e-02, PNorm = 43.5931, GNorm = 0.9133, lr_0 = 6.4376e-04
Validation prc-auc = 0.388926
Epoch 265
Validation prc-auc = 0.388794
Epoch 266
Validation prc-auc = 0.370854
Epoch 267
Validation prc-auc = 0.360171
Epoch 268
Validation prc-auc = 0.335752
Epoch 269
Loss = 2.2726e-02, PNorm = 43.6335, GNorm = 0.3219, lr_0 = 6.3826e-04
Validation prc-auc = 0.321774
Epoch 270
Validation prc-auc = 0.310653
Epoch 271
Validation prc-auc = 0.304311
Epoch 272
Validation prc-auc = 0.303378
Epoch 273
Validation prc-auc = 0.301126
Epoch 274
Loss = 3.0967e-02, PNorm = 43.6661, GNorm = 0.7379, lr_0 = 6.3280e-04
Validation prc-auc = 0.295334
Epoch 275
Validation prc-auc = 0.294356
Epoch 276
Validation prc-auc = 0.303309
Epoch 277
Validation prc-auc = 0.302590
Epoch 278
Validation prc-auc = 0.323177
Epoch 279
Loss = 4.5573e-02, PNorm = 43.6946, GNorm = 0.8849, lr_0 = 6.2739e-04
Validation prc-auc = 0.344107
Epoch 280
Validation prc-auc = 0.347011
Epoch 281
Validation prc-auc = 0.330103
Epoch 282
Validation prc-auc = 0.333556
Epoch 283
Validation prc-auc = 0.349232
Epoch 284
Loss = 6.5798e-02, PNorm = 43.7278, GNorm = 1.4266, lr_0 = 6.2203e-04
Validation prc-auc = 0.351844
Epoch 285
Validation prc-auc = 0.365846
Epoch 286
Validation prc-auc = 0.367456
Epoch 287
Validation prc-auc = 0.364304
Epoch 288
Validation prc-auc = 0.360304
Epoch 289
Loss = 9.7269e-02, PNorm = 43.7650, GNorm = 1.4725, lr_0 = 6.1671e-04
Validation prc-auc = 0.344597
Epoch 290
Validation prc-auc = 0.336240
Epoch 291
Validation prc-auc = 0.330959
Epoch 292
Validation prc-auc = 0.334347
Epoch 293
Validation prc-auc = 0.331407
Epoch 294
Loss = 4.9901e-02, PNorm = 43.8059, GNorm = 0.7538, lr_0 = 6.1144e-04
Validation prc-auc = 0.326223
Epoch 295
Validation prc-auc = 0.323453
Epoch 296
Validation prc-auc = 0.321447
Epoch 297
Validation prc-auc = 0.325222
Epoch 298
Validation prc-auc = 0.326775
Epoch 299
Loss = 5.6153e-02, PNorm = 43.8427, GNorm = 0.9945, lr_0 = 6.0621e-04
Validation prc-auc = 0.328416
Model 0 best validation prc-auc = 0.470326 on epoch 6
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test prc-auc = 0.434991
Ensemble test prc-auc = 0.434991
Fold 9
Splitting data with seed 9
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=1728, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=300, out_features=300, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 873,601
Moving model to cuda
Epoch 0
Validation prc-auc = 0.101711
Epoch 1
Validation prc-auc = 0.166420
Epoch 2
Validation prc-auc = 0.346216
Epoch 3
Validation prc-auc = 0.384411
Epoch 4
Loss = 7.1089e-01, PNorm = 40.7432, GNorm = 4.3725, lr_0 = 6.5000e-04
Validation prc-auc = 0.408894
Epoch 5
Validation prc-auc = 0.397248
Epoch 6
Validation prc-auc = 0.409404
Epoch 7
Validation prc-auc = 0.416173
Epoch 8
Validation prc-auc = 0.407629
Epoch 9
Loss = 5.1558e-01, PNorm = 40.7990, GNorm = 2.0218, lr_0 = 9.9743e-04
Validation prc-auc = 0.408830
Epoch 10
Validation prc-auc = 0.397138
Epoch 11
Validation prc-auc = 0.376245
Epoch 12
Validation prc-auc = 0.362275
Epoch 13
Validation prc-auc = 0.364294
Epoch 14
Loss = 4.8149e-01, PNorm = 40.8614, GNorm = 1.8564, lr_0 = 9.8890e-04
Validation prc-auc = 0.366356
Epoch 15
Validation prc-auc = 0.374564
Epoch 16
Validation prc-auc = 0.375872
Epoch 17
Validation prc-auc = 0.373783
Epoch 18
Validation prc-auc = 0.370708
Epoch 19
Loss = 5.0246e-01, PNorm = 40.9162, GNorm = 2.1803, lr_0 = 9.8045e-04
Validation prc-auc = 0.371793
Epoch 20
Validation prc-auc = 0.373177
Epoch 21
Validation prc-auc = 0.373470
Epoch 22
Validation prc-auc = 0.374986
Epoch 23
Validation prc-auc = 0.380076
Epoch 24
Loss = 5.3625e-01, PNorm = 40.9650, GNorm = 4.8318, lr_0 = 9.7207e-04
Validation prc-auc = 0.385696
Epoch 25
Validation prc-auc = 0.386373
Epoch 26
Validation prc-auc = 0.383877
Epoch 27
Validation prc-auc = 0.389385
Epoch 28
Validation prc-auc = 0.416701
Epoch 29
Loss = 5.3712e-01, PNorm = 41.0107, GNorm = 1.9841, lr_0 = 9.6376e-04
Validation prc-auc = 0.400831
Epoch 30
Validation prc-auc = 0.420914
Epoch 31
Validation prc-auc = 0.428995
Epoch 32
Validation prc-auc = 0.426470
Epoch 33
Validation prc-auc = 0.400106
Epoch 34
Loss = 4.4543e-01, PNorm = 41.0522, GNorm = 1.6474, lr_0 = 9.5552e-04
Validation prc-auc = 0.395555
Epoch 35
Validation prc-auc = 0.384905
Epoch 36
Validation prc-auc = 0.385803
Epoch 37
Validation prc-auc = 0.391218
Epoch 38
Validation prc-auc = 0.390536
Epoch 39
Loss = 5.1166e-01, PNorm = 41.0968, GNorm = 2.2868, lr_0 = 9.4735e-04
Validation prc-auc = 0.380292
Epoch 40
Validation prc-auc = 0.364802
Epoch 41
Validation prc-auc = 0.361245
Epoch 42
Validation prc-auc = 0.364895
Epoch 43
Validation prc-auc = 0.368548
Epoch 44
Loss = 4.3824e-01, PNorm = 41.1483, GNorm = 2.1590, lr_0 = 9.3925e-04
Validation prc-auc = 0.369933
Epoch 45
Validation prc-auc = 0.376889
Epoch 46
Validation prc-auc = 0.388461
Epoch 47
Validation prc-auc = 0.394972
Epoch 48
Validation prc-auc = 0.394445
Epoch 49
Loss = 3.6052e-01, PNorm = 41.2027, GNorm = 1.5571, lr_0 = 9.3122e-04
Validation prc-auc = 0.389760
Epoch 50
Validation prc-auc = 0.392161
Epoch 51
Validation prc-auc = 0.394512
Epoch 52
Validation prc-auc = 0.398539
Epoch 53
Validation prc-auc = 0.401730
Epoch 54
Loss = 3.5929e-01, PNorm = 41.2576, GNorm = 2.1033, lr_0 = 9.2326e-04
Validation prc-auc = 0.413336
Epoch 55
Validation prc-auc = 0.417102
Epoch 56
Validation prc-auc = 0.436856
Epoch 57
Validation prc-auc = 0.437638
Epoch 58
Validation prc-auc = 0.418741
Epoch 59
Loss = 2.4323e-01, PNorm = 41.3116, GNorm = 1.3251, lr_0 = 9.1537e-04
Validation prc-auc = 0.382349
Epoch 60
Validation prc-auc = 0.368495
Epoch 61
Validation prc-auc = 0.362164
Epoch 62
Validation prc-auc = 0.364607
Epoch 63
Validation prc-auc = 0.370868
Epoch 64
Loss = 3.7769e-01, PNorm = 41.3718, GNorm = 1.7330, lr_0 = 9.0754e-04
Validation prc-auc = 0.394550
Epoch 65
Validation prc-auc = 0.390966
Epoch 66
Validation prc-auc = 0.401162
Epoch 67
Validation prc-auc = 0.387687
Epoch 68
Validation prc-auc = 0.355531
Epoch 69
Loss = 3.4113e-01, PNorm = 41.4383, GNorm = 2.0505, lr_0 = 8.9978e-04
Validation prc-auc = 0.336937
Epoch 70
Validation prc-auc = 0.347347
Epoch 71
Validation prc-auc = 0.360329
Epoch 72
Validation prc-auc = 0.417367
Epoch 73
Validation prc-auc = 0.438837
Epoch 74
Loss = 2.2134e-01, PNorm = 41.5069, GNorm = 1.2675, lr_0 = 8.9209e-04
Validation prc-auc = 0.455897
Epoch 75
Validation prc-auc = 0.458217
Epoch 76
Validation prc-auc = 0.424047
Epoch 77
Validation prc-auc = 0.395567
Epoch 78
Validation prc-auc = 0.363791
Epoch 79
Loss = 3.2308e-01, PNorm = 41.5734, GNorm = 2.2616, lr_0 = 8.8447e-04
Validation prc-auc = 0.281936
Epoch 80
Validation prc-auc = 0.349238
Epoch 81
Validation prc-auc = 0.391717
Epoch 82
Validation prc-auc = 0.425216
Epoch 83
Validation prc-auc = 0.443743
Epoch 84
Loss = 2.5431e-01, PNorm = 41.6481, GNorm = 1.5491, lr_0 = 8.7691e-04
Validation prc-auc = 0.451566
Epoch 85
Validation prc-auc = 0.424888
Epoch 86
Validation prc-auc = 0.431117
Epoch 87
Validation prc-auc = 0.428183
Epoch 88
Validation prc-auc = 0.414127
Epoch 89
Loss = 2.0509e-01, PNorm = 41.7185, GNorm = 1.5387, lr_0 = 8.6941e-04
Validation prc-auc = 0.412827
Epoch 90
Validation prc-auc = 0.337679
Epoch 91
Validation prc-auc = 0.324115
Epoch 92
Validation prc-auc = 0.321826
Epoch 93
Validation prc-auc = 0.335207
Epoch 94
Loss = 2.6415e-01, PNorm = 41.7928, GNorm = 2.0740, lr_0 = 8.6198e-04
Validation prc-auc = 0.345330
Epoch 95
Validation prc-auc = 0.386500
Epoch 96
Validation prc-auc = 0.414729
Epoch 97
Validation prc-auc = 0.381964
Epoch 98
Validation prc-auc = 0.379721
Epoch 99
Loss = 2.6723e-01, PNorm = 41.8626, GNorm = 1.7785, lr_0 = 8.5461e-04
Validation prc-auc = 0.346646
Epoch 100
Validation prc-auc = 0.365816
Epoch 101
Validation prc-auc = 0.375809
Epoch 102
Validation prc-auc = 0.379668
Epoch 103
Validation prc-auc = 0.383260
Epoch 104
Loss = 1.2507e-01, PNorm = 41.9326, GNorm = 1.5014, lr_0 = 8.4730e-04
Validation prc-auc = 0.386001
Epoch 105
Validation prc-auc = 0.391342
Epoch 106
Validation prc-auc = 0.394139
Epoch 107
Validation prc-auc = 0.389347
Epoch 108
Validation prc-auc = 0.390013
Epoch 109
Loss = 2.4258e-01, PNorm = 41.9937, GNorm = 1.9806, lr_0 = 8.4006e-04
Validation prc-auc = 0.381198
Epoch 110
Validation prc-auc = 0.372165
Epoch 111
Validation prc-auc = 0.380699
Epoch 112
Validation prc-auc = 0.378749
Epoch 113
Validation prc-auc = 0.378576
Epoch 114
Loss = 2.0778e-01, PNorm = 42.0491, GNorm = 1.4512, lr_0 = 8.3288e-04
Validation prc-auc = 0.375197
Epoch 115
Validation prc-auc = 0.374647
Epoch 116
Validation prc-auc = 0.364127
Epoch 117
Validation prc-auc = 0.337265
Epoch 118
Validation prc-auc = 0.369217
Epoch 119
Loss = 1.3051e-01, PNorm = 42.1021, GNorm = 0.9400, lr_0 = 8.2576e-04
Validation prc-auc = 0.392351
Epoch 120
Validation prc-auc = 0.396552
Epoch 121
Validation prc-auc = 0.363244
Epoch 122
Validation prc-auc = 0.349733
Epoch 123
Validation prc-auc = 0.352640
Epoch 124
Loss = 1.0766e-01, PNorm = 42.1655, GNorm = 1.0308, lr_0 = 8.1870e-04
Validation prc-auc = 0.372542
Epoch 125
Validation prc-auc = 0.375445
Epoch 126
Validation prc-auc = 0.361031
Epoch 127
Validation prc-auc = 0.373866
Epoch 128
Validation prc-auc = 0.371268
Epoch 129
Loss = 1.5039e-01, PNorm = 42.2280, GNorm = 1.1282, lr_0 = 8.1170e-04
Validation prc-auc = 0.356224
Epoch 130
Validation prc-auc = 0.362304
Epoch 131
Validation prc-auc = 0.367309
Epoch 132
Validation prc-auc = 0.392226
Epoch 133
Validation prc-auc = 0.384348
Epoch 134
Loss = 1.5533e-01, PNorm = 42.2895, GNorm = 1.9914, lr_0 = 8.0476e-04
Validation prc-auc = 0.366325
Epoch 135
Validation prc-auc = 0.362248
Epoch 136
Validation prc-auc = 0.371964
Epoch 137
Validation prc-auc = 0.396297
Epoch 138
Validation prc-auc = 0.397523
Epoch 139
Loss = 1.6235e-01, PNorm = 42.3423, GNorm = 2.2738, lr_0 = 7.9788e-04
Validation prc-auc = 0.399548
Epoch 140
Validation prc-auc = 0.387867
Epoch 141
Validation prc-auc = 0.359975
Epoch 142
Validation prc-auc = 0.337237
Epoch 143
Validation prc-auc = 0.333765
Epoch 144
Loss = 1.2220e-01, PNorm = 42.4024, GNorm = 1.2097, lr_0 = 7.9106e-04
Validation prc-auc = 0.334920
Epoch 145
Validation prc-auc = 0.343875
Epoch 146
Validation prc-auc = 0.347288
Epoch 147
Validation prc-auc = 0.351519
Epoch 148
Validation prc-auc = 0.344258
Epoch 149
Loss = 1.4439e-01, PNorm = 42.4673, GNorm = 1.0054, lr_0 = 7.8430e-04
Validation prc-auc = 0.338752
Epoch 150
Validation prc-auc = 0.350726
Epoch 151
Validation prc-auc = 0.365652
Epoch 152
Validation prc-auc = 0.341898
Epoch 153
Validation prc-auc = 0.342662
Epoch 154
Loss = 1.5622e-01, PNorm = 42.5307, GNorm = 1.7379, lr_0 = 7.7759e-04
Validation prc-auc = 0.335827
Epoch 155
Validation prc-auc = 0.325617
Epoch 156
Validation prc-auc = 0.321962
Epoch 157
Validation prc-auc = 0.323894
Epoch 158
Validation prc-auc = 0.369669
Epoch 159
Loss = 6.0595e-02, PNorm = 42.5954, GNorm = 0.4699, lr_0 = 7.7095e-04
Validation prc-auc = 0.397484
Epoch 160
Validation prc-auc = 0.402890
Epoch 161
Validation prc-auc = 0.379846
Epoch 162
Validation prc-auc = 0.370988
Epoch 163
Validation prc-auc = 0.354182
Epoch 164
Loss = 1.0126e-01, PNorm = 42.6543, GNorm = 0.9795, lr_0 = 7.6436e-04
Validation prc-auc = 0.336098
Epoch 165
Validation prc-auc = 0.317829
Epoch 166
Validation prc-auc = 0.321929
Epoch 167
Validation prc-auc = 0.330497
Epoch 168
Validation prc-auc = 0.369130
Epoch 169
Loss = 1.4178e-01, PNorm = 42.7107, GNorm = 1.9344, lr_0 = 7.5782e-04
Validation prc-auc = 0.385827
Epoch 170
Validation prc-auc = 0.386993
Epoch 171
Validation prc-auc = 0.299893
Epoch 172
Validation prc-auc = 0.276662
Epoch 173
Validation prc-auc = 0.263419
Epoch 174
Loss = 1.8389e-01, PNorm = 42.7715, GNorm = 1.5778, lr_0 = 7.5134e-04
Validation prc-auc = 0.265345
Epoch 175
Validation prc-auc = 0.279040
Epoch 176
Validation prc-auc = 0.311069
Epoch 177
Validation prc-auc = 0.341588
Epoch 178
Validation prc-auc = 0.364068
Epoch 179
Loss = 8.2581e-02, PNorm = 42.8299, GNorm = 1.5369, lr_0 = 7.4492e-04
Validation prc-auc = 0.371546
Epoch 180
Validation prc-auc = 0.366699
Epoch 181
Validation prc-auc = 0.363531
Epoch 182
Validation prc-auc = 0.362751
Epoch 183
Validation prc-auc = 0.350525
Epoch 184
Loss = 1.1430e-01, PNorm = 42.8866, GNorm = 0.6266, lr_0 = 7.3855e-04
Validation prc-auc = 0.343212
Epoch 185
Validation prc-auc = 0.325076
Epoch 186
Validation prc-auc = 0.316396
Epoch 187
Validation prc-auc = 0.306190
Epoch 188
Validation prc-auc = 0.296641
Epoch 189
Loss = 1.3419e-01, PNorm = 42.9428, GNorm = 1.6789, lr_0 = 7.3224e-04
Validation prc-auc = 0.273817
Epoch 190
Validation prc-auc = 0.285679
Epoch 191
Validation prc-auc = 0.344212
Epoch 192
Validation prc-auc = 0.360631
Epoch 193
Validation prc-auc = 0.376428
Epoch 194
Loss = 8.5570e-02, PNorm = 43.0034, GNorm = 1.1763, lr_0 = 7.2598e-04
Validation prc-auc = 0.379348
Epoch 195
Validation prc-auc = 0.390055
Epoch 196
Validation prc-auc = 0.392035
Epoch 197
Validation prc-auc = 0.375548
Epoch 198
Validation prc-auc = 0.350492
Epoch 199
Loss = 6.2023e-02, PNorm = 43.0583, GNorm = 0.9088, lr_0 = 7.1977e-04
Validation prc-auc = 0.327977
Epoch 200
Validation prc-auc = 0.312367
Epoch 201
Validation prc-auc = 0.257334
Epoch 202
Validation prc-auc = 0.251749
Epoch 203
Validation prc-auc = 0.250233
Epoch 204
Loss = 1.0093e-01, PNorm = 43.1210, GNorm = 1.8133, lr_0 = 7.1362e-04
Validation prc-auc = 0.253749
Epoch 205
Validation prc-auc = 0.302523
Epoch 206
Validation prc-auc = 0.320638
Epoch 207
Validation prc-auc = 0.329745
Epoch 208
Validation prc-auc = 0.349426
Epoch 209
Loss = 6.6048e-02, PNorm = 43.1712, GNorm = 1.3779, lr_0 = 7.0752e-04
Validation prc-auc = 0.362690
Epoch 210
Validation prc-auc = 0.371881
Epoch 211
Validation prc-auc = 0.382794
Epoch 212
Validation prc-auc = 0.385661
Epoch 213
Validation prc-auc = 0.365085
Epoch 214
Loss = 9.3397e-02, PNorm = 43.2223, GNorm = 1.1101, lr_0 = 7.0147e-04
Validation prc-auc = 0.337486
Epoch 215
Validation prc-auc = 0.303259
Epoch 216
Validation prc-auc = 0.289505
Epoch 217
Validation prc-auc = 0.276678
Epoch 218
Validation prc-auc = 0.281893
Epoch 219
Loss = 1.1686e-01, PNorm = 43.2673, GNorm = 2.1579, lr_0 = 6.9548e-04
Validation prc-auc = 0.296426
Epoch 220
Validation prc-auc = 0.346298
Epoch 221
Validation prc-auc = 0.377473
Epoch 222
Validation prc-auc = 0.389259
Epoch 223
Validation prc-auc = 0.401278
Epoch 224
Loss = 1.0926e-01, PNorm = 43.3266, GNorm = 0.5220, lr_0 = 6.8953e-04
Validation prc-auc = 0.417948
Epoch 225
Validation prc-auc = 0.428812
Epoch 226
Validation prc-auc = 0.427075
Epoch 227
Validation prc-auc = 0.427382
Epoch 228
Validation prc-auc = 0.411946
Epoch 229
Loss = 5.6039e-02, PNorm = 43.3729, GNorm = 0.7669, lr_0 = 6.8364e-04
Validation prc-auc = 0.421687
Epoch 230
Validation prc-auc = 0.400187
Epoch 231
Validation prc-auc = 0.396589
Epoch 232
Validation prc-auc = 0.386601
Epoch 233
Validation prc-auc = 0.372528
Epoch 234
Loss = 7.1882e-02, PNorm = 43.4126, GNorm = 1.3335, lr_0 = 6.7779e-04
Validation prc-auc = 0.352793
Epoch 235
Validation prc-auc = 0.339107
Epoch 236
Validation prc-auc = 0.336055
Epoch 237
Validation prc-auc = 0.329674
Epoch 238
Validation prc-auc = 0.325477
Epoch 239
Loss = 1.7470e-01, PNorm = 43.4551, GNorm = 3.7033, lr_0 = 6.7200e-04
Validation prc-auc = 0.315291
Epoch 240
Validation prc-auc = 0.324941
Epoch 241
Validation prc-auc = 0.320192
Epoch 242
Validation prc-auc = 0.304176
Epoch 243
Validation prc-auc = 0.295637
Epoch 244
Loss = 5.9404e-02, PNorm = 43.5058, GNorm = 0.7333, lr_0 = 6.6625e-04
Validation prc-auc = 0.294173
Epoch 245
Validation prc-auc = 0.301430
Epoch 246
Validation prc-auc = 0.311835
Epoch 247
Validation prc-auc = 0.317726
Epoch 248
Validation prc-auc = 0.326455
Epoch 249
Loss = 1.1436e-01, PNorm = 43.5567, GNorm = 2.0013, lr_0 = 6.6056e-04
Validation prc-auc = 0.316951
Epoch 250
Validation prc-auc = 0.319749
Epoch 251
Validation prc-auc = 0.323970
Epoch 252
Validation prc-auc = 0.340309
Epoch 253
Validation prc-auc = 0.362092
Epoch 254
Loss = 5.5623e-02, PNorm = 43.6068, GNorm = 0.3686, lr_0 = 6.5491e-04
Validation prc-auc = 0.361233
Epoch 255
Validation prc-auc = 0.356656
Epoch 256
Validation prc-auc = 0.351593
Epoch 257
Validation prc-auc = 0.342753
Epoch 258
Validation prc-auc = 0.338868
Epoch 259
Loss = 4.5771e-02, PNorm = 43.6532, GNorm = 0.5268, lr_0 = 6.4931e-04
Validation prc-auc = 0.337120
Epoch 260
Validation prc-auc = 0.337319
Epoch 261
Validation prc-auc = 0.335845
Epoch 262
Validation prc-auc = 0.333538
Epoch 263
Validation prc-auc = 0.336077
Epoch 264
Loss = 1.1674e-01, PNorm = 43.6974, GNorm = 0.4473, lr_0 = 6.4376e-04
Validation prc-auc = 0.328611
Epoch 265
Validation prc-auc = 0.326853
Epoch 266
Validation prc-auc = 0.341372
Epoch 267
Validation prc-auc = 0.347342
Epoch 268
Validation prc-auc = 0.349910
Epoch 269
Loss = 1.4459e-01, PNorm = 43.7349, GNorm = 1.2311, lr_0 = 6.3826e-04
Validation prc-auc = 0.350098
Epoch 270
Validation prc-auc = 0.348340
Epoch 271
Validation prc-auc = 0.336629
Epoch 272
Validation prc-auc = 0.331747
Epoch 273
Validation prc-auc = 0.331960
Epoch 274
Loss = 4.8250e-02, PNorm = 43.7721, GNorm = 1.2701, lr_0 = 6.3280e-04
Validation prc-auc = 0.332724
Epoch 275
Validation prc-auc = 0.340897
Epoch 276
Validation prc-auc = 0.338480
Epoch 277
Validation prc-auc = 0.321936
Epoch 278
Validation prc-auc = 0.319057
Epoch 279
Loss = 4.4517e-02, PNorm = 43.8139, GNorm = 0.3651, lr_0 = 6.2739e-04
Validation prc-auc = 0.318909
Epoch 280
Validation prc-auc = 0.318833
Epoch 281
Validation prc-auc = 0.325381
Epoch 282
Validation prc-auc = 0.320056
Epoch 283
Validation prc-auc = 0.314275
Epoch 284
Loss = 6.5541e-02, PNorm = 43.8540, GNorm = 1.9533, lr_0 = 6.2203e-04
Validation prc-auc = 0.316078
Epoch 285
Validation prc-auc = 0.319878
Epoch 286
Validation prc-auc = 0.334379
Epoch 287
Validation prc-auc = 0.338213
Epoch 288
Validation prc-auc = 0.352721
Epoch 289
Loss = 4.7104e-02, PNorm = 43.8941, GNorm = 1.4148, lr_0 = 6.1671e-04
Validation prc-auc = 0.341652
Epoch 290
Validation prc-auc = 0.335895
Epoch 291
Validation prc-auc = 0.337461
Epoch 292
Validation prc-auc = 0.338837
Epoch 293
Validation prc-auc = 0.346191
Epoch 294
Loss = 7.8974e-02, PNorm = 43.9347, GNorm = 1.2560, lr_0 = 6.1144e-04
Validation prc-auc = 0.370388
Epoch 295
Validation prc-auc = 0.379490
Epoch 296
Validation prc-auc = 0.387005
Epoch 297
Validation prc-auc = 0.386221
Epoch 298
Validation prc-auc = 0.362413
Epoch 299
Loss = 4.8535e-02, PNorm = 43.9699, GNorm = 1.0519, lr_0 = 6.0621e-04
Validation prc-auc = 0.350026
Model 0 best validation prc-auc = 0.458217 on epoch 75
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test prc-auc = 0.444722
Ensemble test prc-auc = 0.444722
10-fold cross validation
	Seed 0 ==> test prc-auc = 0.399693
	Seed 1 ==> test prc-auc = 0.471997
	Seed 2 ==> test prc-auc = 0.378929
	Seed 3 ==> test prc-auc = 0.439390
	Seed 4 ==> test prc-auc = 0.404111
	Seed 5 ==> test prc-auc = 0.422134
	Seed 6 ==> test prc-auc = 0.484414
	Seed 7 ==> test prc-auc = 0.469799
	Seed 8 ==> test prc-auc = 0.434991
	Seed 9 ==> test prc-auc = 0.444722
Overall test prc-auc = 0.435018 +/- 0.032627
Elapsed time = 0:08:37
